JPWO2020071559A1 - Vehicle condition judgment device, its judgment program and its judgment method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To determine a damaged state of a vehicle flexibly, at high speed and with high accuracy. A component determination unit 4 determines a vehicle component in a captured image with reference to a component learning model 9. The parts learning model 9 is constructed by supervised learning using data related to vehicle parts as supervised data by using an object detection algorithm by deep learning. The object detection algorithm inputs the captured image into a single neural network system, and uses a regression-problem approach to collectively extract the region of the vehicle part in the captured image with attribute classification. The state determination unit 5 determines the damaged state of the vehicle parts determined by the component determination unit 4 with reference to the state learning model 10. The state learning model 10 is constructed by supervised learning using data on the damaged state of vehicle parts as teacher data.

Description

The present invention relates to a vehicle state determination device for determining a vehicle damage state from a captured image, a determination program thereof, and a determination method.

For example, Patent Document 1 discloses an accident vehicle repair cost estimation system that can accurately and quickly evaluate damage to an accident vehicle and estimate repairs. This estimation system includes a capture means, a storage means, an input means, a display means, a link means, and an estimation means. The capturing means captures the image data of the accident vehicle. The storage means stores the vehicle attribute data necessary for estimating the accident vehicle repair cost. The input means inputs the estimation data necessary for estimating the repair cost of the accident vehicle. The display means displays various data including accident vehicle image data. The linking means determines which part of the vehicle the image data clearly indicates damage. The estimation means simultaneously displays the image data and the vehicle attribute data of the portion corresponding to the image data on the display means. In addition, the estimation means performs estimation processing of the cost required for repairing the accident vehicle based on the estimation data and the vehicle attribute data.

Japanese Unexamined Patent Publication No. 10-197285

However, in the system of Patent Document 1, the degree of damage is determined from the closeness of the image data of the accident vehicle to the image data of the vehicle repaired in the past. Therefore, if there is no appropriate past image data, Judgment accuracy is reduced.

Therefore, an object of the present invention is to determine the damaged state of the vehicle flexibly, at high speed and with high accuracy.

In order to solve such a problem, the first invention provides a vehicle state determination device having a component determination unit and a state determination unit, and determining a damaged state of the vehicle from a captured image. The parts determination unit determines the vehicle parts in the captured image with reference to the parts learning model. The parts learning model is constructed by supervised learning using data related to vehicle parts as supervised data using an object detection algorithm by deep learning. The object detection algorithm inputs the captured image into a single neural network system, and uses a regression-problem approach to collectively extract the region of the vehicle part in the captured image with attribute classification. The state determination unit determines the damaged state of each vehicle part determined by the parts determination unit with reference to the state learning model. The state learning model is constructed by supervised learning using data on the damaged state of vehicle parts as supervised data.

Here, in the first invention, the object detection algorithm is preferably YOLO or SSD. Further, in the first invention, a surface determination unit may be further provided. The surface determination unit determines the constituent surface of the vehicle in the captured image with reference to the surface learning model. The surface learning model is constructed by supervised learning using data on the constituent surfaces of the vehicle as teacher data. In this case, it is preferable that the component determination unit filters the vehicle parts based on the constituent surface determined by the surface determination unit, and outputs the filtered vehicle parts as the determination result. Further, when a vehicle component unique to a specific constituent surface is present in the captured image, the surface determination unit may include the constituent surface corresponding to the specific constituent surface in the determination result.

In the first invention, an estimate calculation unit may be further provided. The estimation calculation unit estimates the repair cost of the vehicle from the damaged state of each vehicle part determined by the condition determination unit by referring to the estimation table. The estimation table holds the repair cost associated with the damaged state of the vehicle parts for each vehicle part.

In the first invention, at least one of the transition of the process from the surface determination unit to the component determination unit, the transfer of the process from the component determination unit to the state determination unit, and the transfer of the process from the state determination unit to the estimate calculation unit. One is preferably performed on the condition that the user's approval is obtained while allowing the user to modify the determination result.

The second invention provides a vehicle state determination program that causes a computer to execute a process having a component determination step and a state determination step, and determines a damaged state of the vehicle from an captured image. In the component determination step, the vehicle component in the captured image is determined with reference to the component learning model. The parts learning model is constructed by supervised learning using data related to vehicle parts as supervised data using an object detection algorithm by deep learning. The object detection algorithm inputs the captured image into a single neural network system, and uses a regression-problem approach to collectively extract the region of the vehicle part in the captured image with attribute classification. In the state determination step, the damage state of each vehicle part determined by the parts determination unit is determined with reference to the state learning model. The state learning model is constructed by supervised learning using data on the damaged state of vehicle parts as supervised data.

The third invention provides a vehicle state determination method that includes a parts determination step and a state determination step, and determines a vehicle damage state from a captured image. In the component determination step, the vehicle component in the captured image is determined with reference to the component learning model. The parts learning model is constructed by supervised learning using data related to vehicle parts as supervised data using an object detection algorithm by deep learning. The object detection algorithm inputs the captured image into a single neural network system, and uses a regression-problem approach to collectively extract the region of the vehicle part in the captured image with attribute classification. In the state determination step, the damage state of each vehicle part determined by the parts determination unit is determined with reference to the state learning model. The state learning model is constructed by supervised learning using data on the damaged state of vehicle parts as supervised data.

Here, in the second and third inventions, the object detection algorithm is preferably YOLO or SSD. Further, in the second and third inventions, a surface determination step may be further provided. In the surface determination step, the constituent surface of the vehicle in the captured image is determined with reference to the surface learning model. The surface learning model is constructed by supervised learning using data on the constituent surfaces of the vehicle as teacher data. In this case, it is preferable that the component determination step filters the vehicle parts based on the constituent surface determined by the surface determination step, and outputs the filtered vehicle parts as the determination result. Further, in the surface determination step, when a vehicle component unique to a specific constituent surface is present in the captured image, the constituent surface corresponding to the specific constituent surface may be included in the determination result.

In the second and third inventions, an estimate calculation step may be further provided. In the estimation calculation step, the repair cost of the vehicle is estimated from the damaged state of each vehicle part determined by the state determination step by referring to the estimation table. The estimation table holds the repair cost associated with the damaged state of the vehicle parts for each vehicle part.

In the second and third inventions, the process shifts from the surface determination step to the component determination step, the process shifts from the component determination step to the state determination step, and the process shifts from the state determination step to the estimate calculation step. It is preferable that at least one of them is performed on condition that the user's approval is obtained while allowing the user to modify the determination result.

According to the present invention, by determining the damaged state of a vehicle based on machine learning, it is possible to flexibly deal with various vehicles including an unknown vehicle. At that time, the processing is separated into the component determination and the damage determination, and the damage determination of each component is performed after the component determination is performed, so that the determination accuracy as a whole can be improved. In addition, for component determination in which many vehicle parts can be detected as objects in the captured image, as represented by YOLO and SSD, the captured image is input to a single neural network system, and a regression problem approach is used. By adopting an object detection algorithm that collectively extracts the area of vehicle parts with attribute classification, it is possible to speed up the processing.

Block diagram of vehicle condition judgment device Figure showing screen display example of image reception The figure which shows the screen display example of the surface judgment result Explanatory diagram of object detection algorithm YOLO network configuration diagram The figure which shows the screen display example of the vehicle part judgment result The figure which shows the screen display example of the judgment result of the damage state Schematic block diagram of the quotation table Figure showing screen display example of estimation result

FIG. 1 is a block diagram of the vehicle state determination device 1 according to the present embodiment. The vehicle condition determination device 1 determines the damaged state of the vehicle from the captured image designated by the user, and presents the estimated cost required for repairing the vehicle to the user. The vehicle to be determined is assumed to be a private vehicle in this embodiment, but this is an example, and any vehicle including trucks, buses, motorcycles, etc. should be targeted depending on the design specifications. Can be done. The vehicle state determination device 1 can be equivalently realized by installing a computer program (form layout analysis program) on the computer.

The vehicle state determination device 1 includes an image reception unit 2, a surface determination unit 3, a parts determination unit 4, a state determination unit 5, an estimate calculation unit 6, an input / output interface 7, a learning model 8 to 10. It is mainly composed of the estimation table 11. Each processing unit 2 to 6 is connected to a display device 12 via an input / output interface 7, and the input / output interface 7 controls input / output between these and the display device 12. Basically, each process in the processing units 3 to 6 is sequentially performed, but the transfer of the process between adjacent processing units, specifically, the transfer from the surface determination unit 3 to the part determination unit 4, and the component determination. The transition from the unit 4 to the status determination unit 5 and the transition from the status determination unit 5 to the estimate calculation unit 6 are subject to the user's approval after allowing the user to correct the determination result. Will be done. This is to improve the judgment accuracy as a whole by appropriately reflecting the user's intention in the processing process. However, some of the process transitions may be done automatically without the condition of user approval. When the display device 12 is connected to the Internet or the like via a network, the input / output interface 7 has a communication function necessary for performing network communication.

The image receiving unit 2 receives an captured image to be determined from the display device 12, specifically, an image captured by a camera of the appearance of the damaged vehicle. The direction in which the vehicle is imaged may be any of front, rear, and sideways, and may be diagonally forward or diagonally rearward.

The captured image to be determined is designated by the user operating the display device 12 while looking at the screen, and is output / uploaded to the vehicle state determination device 1. FIG. 2 is a diagram showing an example of screen display of an image reception in the display device 12. The display screen 30 for receiving an image has an image receiving area 31. In the image receiving area 31, thumbnails of captured images to be determined are displayed. The user specifies a determination target by designating a specific image file through the file reference button or by dropping a specific image file in the image receiving area 31. As can be understood from the existence of a plurality of blank frames in the image receiving area 31, a plurality of determination targets can be specified, and even if the determination target has already been specified, it can be canceled through the cancel button. When the specification of the judgment target is completed, the user presses the judgment start button. By this action, the determination target is output to the image receiving unit 2. The captured image received by the image receiving unit 2 is output to the surface determination unit 3.

The captured image to be determined may be a color image, but a grayscale image may be accepted in order to reduce the memory usage. The size of the captured image is appropriately set in consideration of the memory usage of the system and the like.

The surface determination unit 3 determines the constituent surface of the vehicle projected on the captured image to be determined. Here, the "construction surface" refers to an individual surface that constitutes a three-dimensional vehicle, and includes the front surface, the back surface, and the side surface of the vehicle. For example, in the case of a captured image of the vehicle taken from the front, the determination result is the front side, and in the case of the captured image of the vehicle taken from the rear, the determination result is the back side. However, the number of constituent surfaces obtained as a determination result is not limited to one, and may be multiple. For example, in the case of an captured image of a vehicle taken diagonally from the front, the determination results are front and side surfaces. The reason for determining the configuration surface in this way is to improve the determination accuracy in the post-processing.

The surface learning model 8 is referred to in the determination of the constituent surface. This surface learning model 8 is mainly composed of a neural network system that imitates the human cranial nerves. The neural network system is formed on the work area of a computer and has an input layer, a hidden layer, and an output layer. The input layer is weighted by an activation function when transmitting an input signal to the hidden layer. Then, the transmission with weighting according to the number of layers of the hidden layer is repeated, and the signal transmitted to the output layer is finally output. The number of ports in the input layer, the number of ports in the output layer, the number of layers in the hidden layer, etc. are arbitrary. The output layer also outputs the classification probabilities of the outputs (front, back, sides, etc.).

The surface learning model 8 is constructed by supervised learning using data related to the constituent surfaces of the vehicle as teacher data. The teacher data has a captured image of the vehicle and a classification of the constituent surfaces of the vehicle, and various and large amounts of data are used including various vehicle types, various body colors, various shooting directions, and the like. In supervised learning, the classification probability of the output with respect to the input of supervised data is verified, and the surface learning model 8 is set to a desired state by repeating the adjustment of the activation function (weighting) based on this.

As the surface learning model 8, in addition to the neural network, a machine learning method such as a support vector machine, a decision tree, a Bayesian network, a linear regression, a multivariate analysis, a logistic regression analysis, or a judgment analysis may be used. Further, a convolutional neural network and R-CCN (Regions with CNN features) using the convolutional neural network may be used. This point is the same for the state learning model 10 described later.

Further, in the determination of the constituent surface, in addition to the reference of the surface learning model 8, the constituent surface may be estimated by paying attention to the presence or absence of the unique vehicle parts existing in the specific constituent surface. For example, the front light and the front grille are unique to the front surface of the vehicle and have a characteristic shape that can be distinguished from other vehicle parts. Therefore, due to the presence of the front light and the like, at least the front surface is included in the captured image. It can be considered to be included. For this reason, when a vehicle component unique to a specific constituent surface is present in the captured image, the constituent surface corresponding to the unique constituent surface is included in the determination result.

Further, the surface learning model 8 may be reduced in weight in order to suppress the memory usage of the GPU (Graphics Processing Unit) or to improve the learning efficiency. As an example, the number of VGG blocks (1 block by convolution-> convolution-> pooling) in a general-purpose model such as VGG-16 can be reduced, and the size of the captured image can be reduced. VGG-16 is a method of using an already trained model for a CNN (convolutional neural network), that is, one of the transfer learning models, which includes a total of 16 layers including a convolution layer and a fully connected layer, and has a large convolution filter. All of them are 3x3, and the fully connected layer consists of 2 layers of 4096 units and 1 layer of 1000 units for classification.

The determination result of the surface determination unit 3 is output to the display device 12. FIG. 3 is a diagram showing a screen display example of the surface determination result displayed on the display device 12. The determination result display screen 40 has an image display area 41, a plurality of determination result display areas 42, and a plurality of check boxes 43. In the image receiving area 41, a captured image of the vehicle related to the determination target is displayed. In each determination result display area 42, candidates (front surface, side surface, back surface) of the constituent surface of the vehicle are displayed with a classification probability. The check boxes 43 are provided corresponding to the respective determination result display areas 42, and those satisfying a predetermined condition are marked with a check mark. The predetermined conditions are that the classification probability of the constituent surface candidates is equal to or higher than the predetermined threshold value (in this case, a plurality of check marks may be added), and the constituent surface classification probability is the highest. And so on. When the user determines that the determination result is valid, the user presses the determination continuation button. If the user determines that this is not appropriate, he / she corrects the determination result including the change of the tick mark of the check box 43, and then presses the determination continuation button. By this action, the determination result of the surface determination unit 3 (including the determination result corrected by the user) is output to the component determination unit 4 together with the captured image.

The determination result modified by the user may be reflected in the surface learning model 8. Thereby, the learning depth in the determination of the constituent surface can be deepened.

The component determination unit 4 extracts the vehicle parts projected on the captured image, specifically, the component area on which the vehicle parts are projected and the attributes of the vehicle parts individually. In this component determination, the component learning model 9 is referred to. This learning model 9 is constructed based on an object detection algorithm by deep learning such as YOLO and SSD in consideration of multi-scale property and operation speed.

FIG. 4 is an explanatory diagram of the object detection algorithm. As shown in FIG. 6A, the conventional detection method used for face detection and the like is divided into three stages of input processing: area search, feature extraction, and machine learning. That is, first, the area search is performed, then the feature extraction is performed according to the object to be detected, and finally the appropriate machine learning method is selected. In this detection method, object detection is realized by dividing it into three algorithms. As for the feature quantity, basically, only a specific target can be detected because it is designed exclusively for the detection target. Therefore, in order to eliminate such restrictions, an object detection algorithm by deep learning as shown in FIGS. (B) and (c) has been proposed. As shown in FIG. 3B, in R-CNN and the like, feature extraction is automatically realized by using deep learning. This allows flexible classification of various objects simply by designing the network. However, the area search still remains as a separate process. Therefore, the method shown in Fig. (C) represented by YOLO (You Only Look Once) and SSD (Single Shot MultiBox Detector) includes the area search in the deep learning. In this method, by inputting an input (captured image) to a single neural network, extraction of an item area and classification of its attributes are performed collectively. The first feature of this method is that it is a regression problem approach. Regression is an approach that directly predicts numerical values from the tendency of data, and instead of deciding a region and then classifying what it is, the coordinates and size of an object are directly predicted. Second, the process is completed in a single network. It can also be called "End-to-End" processing in the sense that after data is input, it goes to the end (output result) only by deep learning. One of the features of this embodiment is that the method of FIG. 3C represented by YOLO or SSD is adopted for determining parts in which many vehicle parts can be detected as objects in the captured image. ..

For example, the processing of YOLO is as follows. First, the input image is divided into S × S regions. Next, the classification probabilities of the objects in each region are derived. Then, the parameters (x, y, height, width) and the confidence (confidence) of B bounding boxes (hyperparameters) are calculated. The bounding box is the circumscribed quadrangle of the object area, and the reliability is the degree of agreement between the predicted and the correct bounding box. For object detection, the product of the classification probability of the object and the reliability of each bounding box is used. FIG. 5 is a network configuration diagram of YOLO. In YOLO, the form image is input to the CNN (Convolutional Neural Network) layer, and the result is output through the fully connected layers of a plurality of stages. The output includes an image area divided into S * S pieces, five parameters of a bounding box (BB) including reliability (classification accuracy), and the number of classes (item attributes).

The parts learning model 9 is constructed by supervised learning using data related to vehicle parts for each constituent surface of the vehicle as teacher data. This teacher data has a partial image of a vehicle part and an attribute of the vehicle part, and various and large amounts of data are used including various vehicle types, various vehicle parts, various shooting directions, and the like. In order to secure a large amount of data, a source image that has undergone image processing is also used. However, in order to distinguish between the right headlamp and the left headlamp, the image is not flipped horizontally, which is one of the image processing. By using such a parts learning model 9, it is possible to determine and extract vehicle parts included in the constituent surface of the vehicle regardless of the presence or absence of damage to the vehicle and its degree.

The parts determination unit 4 filters the vehicle parts specified based on the parts learning model 9 based on the constituent surfaces of the vehicle determined by the surface determination unit 3, and outputs the filtered vehicle parts as the determination result. For example, when the determination result of the constituent surface is the front surface and the side surface, it is clear that this is an erroneous determination even if the tail lamp is extracted as a result of the component determination. This is because the tail lamps are on the back and cannot be on the front and sides. Such correlation is also recognized for side doors unique to the sides, headlights specific to the front, front grilles, front windows, and the like. Therefore, among the vehicle parts obtained as a result of the parts judgment, only those related to the constituent surface obtained as the judgment result of the constituent surface are included in the judgment result, and the others are excluded to improve the parts determination accuracy. Can be planned.

The determination result of the component determination unit 4 is output to the display device 12. FIG. 6 is a diagram showing a screen display example of the component determination result. The determination result display screen 50 has an image display area 51, a plurality of determination result display areas 52, and a plurality of check boxes 53. In the image display area 51, a captured image of the vehicle related to the determination target is displayed, and a rectangular frame showing individual vehicle parts extracted by the component determination unit 4 is displayed. In the determination result display area 52, candidates for each vehicle component (right headlight, left headlight, front window, etc.) are displayed with a classification probability. The check boxes 53 are provided corresponding to the respective determination result display areas 52, and those satisfying a predetermined condition are marked with a check mark. As a predetermined condition, typically, the classification probability of the candidate of the constituent surface is equal to or higher than the predetermined threshold value. When the user determines that the determination result is valid, the user presses the determination continuation button. If the user determines that this is not appropriate, he / she corrects the determination result including the change of the tick mark of the check box 53, and then presses the determination continuation button. By this action, the determination result of the component determination unit 4 (including the determination result corrected by the user) is output to the state determination unit 5 together with the captured image.

The determination result corrected by the user may be reflected in the component learning model 9. As a result, the learning depth of component determination can be deepened.

The state determination unit 5 determines the damaged state of each vehicle part extracted by the part determination unit 4. This damage state determination is performed by referring to the state learning model 10 constructed by supervised learning using the teacher data regarding the damage state of the vehicle parts. The configuration of the state learning model 10 basically conforms to that of the surface learning model 8.

Further, the construction of the state learning model 10 is performed by supervised learning using data on the damaged state of the vehicle parts as teacher data. This teacher data includes a partial image of a damaged vehicle part and attributes that classify the damaged state (for example, "replacement", "detachment", damage degree "large", "medium", "small"). It has a large amount of diverse data, including various vehicle types, various vehicle parts, and various shooting directions. Here, "replacement" means damage to the extent that the vehicle parts themselves must be replaced rather than being repaired. "Detachment" means that the vehicle part itself is undamaged, but must be removed from the vehicle body in order to replace or repair other damaged vehicle parts. In addition, in order to secure a large amount of data, a certain source image that has undergone image processing may be used. By determining the damaged state for each vehicle part using such a state learning model 10, it is possible to improve the accuracy of determining the damaged state.

In the present embodiment, the state determination unit 5 uses a network in which only the fully connected layer is replaced based on the general-purpose model such as the VGG-16 described above. Further, since the amount of data varies widely for each vehicle part, fine tuning may be performed by the weight learned in advance by ImageNet in order to perform learning even with a small amount of data. Fine tuning is a method of constructing a new model by reusing a part of an existing model. Furthermore, in order to deal with the case where an image that is slightly pulled for each data and an image of a different type from the up image are mixed, the up image may be selected and cleansing may be performed by trimming the target portion (trimming). Some of the images that have been cropped are also used).

The determination result of the state determination unit 5 is output to the display device 12. FIG. 7 is a diagram showing a screen display example of the damage state determination result displayed on the display device 12. The determination result display screen 60 has an image display area 61, a plurality of determination result display areas 62, and a plurality of check boxes 63. In the image display area 61, a captured image of the vehicle related to the determination target is displayed, and a round frame indicating the damaged portion determined by the state determination unit 5 is displayed. In the determination result display area 62, candidates for the damaged state (replacement, desorption, large, medium, small) are displayed with a classification probability. The check boxes 63 are provided corresponding to the respective determination result display areas 62, and those satisfying a predetermined condition are marked with a check mark. A predetermined condition typically includes a probability of classification of damage degree candidates greater than or equal to a predetermined threshold. When the user determines that the determination result is appropriate, the user presses the estimate start continuation button. If the user determines that this is not appropriate, he / she corrects the determination result including the change of the tick mark of the check box 63, and then presses the estimation start button. By this action, the determination result of the state determination unit 5 (including the determination result corrected by the user) is output to the estimation calculation unit 6 together with the captured image.

The determination result corrected by the user may be reflected in the state learning model 10. As a result, the learning depth for determining the damage state can be deepened.

The estimate calculation unit 6 estimates the cost required for repairing the vehicle related to the determination target by referring to the estimate table 11. FIG. 8 is a schematic configuration diagram of the estimation table 11. In this estimation table 11, the vehicle part name, the degree of damage, and the cost including wages are stored in association with each other. The cost is specified for each vehicle part by searching the estimation table 11 using the vehicle parts specified by the parts determination unit 4 and the damaged state of the vehicle parts determined by the state determination unit 5 as keys. The cost required to repair the vehicle is the total of the costs for each vehicle part. For example, when the parts determination unit 4 extracts the "front bumper" and the "headlight", and the state determination unit 5 determines that the degree of damage to the front vehicle is "medium" and the degree of damage to the latter is "small", the former The wage will be "\ 200,000", the latter wage will be "\ 30,000", and the total amount will be "\ 230,000".

If the design specifications are such that the vehicle type is specified by the analysis of the captured image, or the vehicle type is specified by the user, and the cost is calculated on the assumption that the vehicle type is known, the figure is shown in the figure. As shown in the above, the vehicle type (for example, “ABC”) may be included as an item in the estimation table 11. In this way, by setting the cost in detail for each vehicle type, it is possible to accurately calculate the estimated amount according to the vehicle type.

The estimation result of the estimation calculation unit 6 is output to the display device 12. FIG. 9 is a diagram showing a screen display example of the estimation result displayed on the display device 12. The estimation result display screen 70 has an image display area 71 and an estimation result display area 72. In the image display area 71, a captured image of the vehicle related to the determination target is displayed. In the estimation result display area 72, the damaged vehicle parts (parts), the degree of damage, and the cost are displayed. If there are multiple damaged vehicle parts, the total cost will be displayed along with the individual costs. When the user presses the end button, a series of processes in the vehicle state determination device 1 are completed.

As described above, according to the present embodiment, by determining the damaged state of the vehicle based on machine learning, it is possible to flexibly deal with various vehicles including an unknown vehicle. At that time, the processing is separated by the parts determination unit 4 and the state determination unit 5, and after the parts are determined, the damage state is determined for each vehicle part, so that these can be processed by a single machine learning. In comparison, it is possible to improve the judgment accuracy as a whole.

Further, according to the present embodiment, for the component determination in which many vehicle parts can be detected as objects in the captured image, the captured image is input to a single neural network system as represented by YOLO and SSD. , Adopt an object detection algorithm that collectively extracts the area of vehicle parts with attribute classification by a regression problematic approach. This makes it possible to speed up the processing in the component determination unit 4.

Further, according to the present embodiment, prior to the processing of the component determination unit 4, the surface determination unit 2 identifies the constituent surface of the vehicle in the captured image, and then executes the component determination for the captured image. As a result, among the vehicle parts obtained as the determination result of the component determination unit 4, those that cannot exist on the constituent surface specified by the surface determination unit 2 can be excluded as erroneous determination, so that the accuracy of component determination is improved. Can be planned.

Further, according to the present embodiment, the cost required for repairing the vehicle is presented to the user by providing the estimation calculation unit 6 for estimating the repair cost of the vehicle from the damaged state of each vehicle part determined by the state determination unit 5. do. As a result, the convenience of the user can be further enhanced.

1 Vehicle condition judgment device 2 Image reception unit 3 Surface judgment unit 4 Parts judgment unit 5 Condition judgment unit 6 Estimate calculation unit 7 Input / output interface 8 Surface learning model 9 Parts learning model 10 State learning model 11 Estimate table 12 Display device

Claims (18)

In the vehicle condition determination device that determines the damaged condition of the vehicle from the captured image,
A parts determination unit that determines vehicle parts in a captured image by referring to a parts learning model constructed by supervised learning using data related to vehicle parts as supervised learning using an object detection algorithm by deep learning.
It has a state determination unit that determines the damage state of each vehicle part determined by the parts determination unit with reference to a state learning model constructed by supervised learning using data on the damage state of vehicle parts as teacher data. death,
The object detection algorithm
By inputting the captured image into a single neural network system, the vehicle state determination is characterized in that the region extraction of the vehicle parts in the captured image is collectively performed with the classification of attributes by a regression problem approach. Device. The vehicle state determination device according to claim 1, wherein the object detection algorithm is YOLO or SSD. It also has a surface determination unit that determines the constituent surface of the vehicle in the captured image by referring to the surface learning model constructed by supervised learning using the data related to the constituent surface of the vehicle as teacher data.
The first aspect of claim 1, wherein the component determination unit filters the vehicle parts based on the constituent surface determined by the surface determination unit, and outputs the filtered vehicle parts as a determination result. Vehicle condition determination device. The surface determination unit according to claim 3, wherein when a vehicle component unique to a specific constituent surface is present in the captured image, the surface determination unit includes the constituent surface corresponding to the specific constituent surface in the determination result. Vehicle condition determination device. By referring to the estimation table in which the repair cost associated with the damaged state of the vehicle part is held for each vehicle part, the repair cost of the vehicle is estimated from the damaged state of each vehicle part determined by the state determination unit. The vehicle condition determination device according to any one of claims 1 to 4, further comprising an estimate calculation unit. At least one of the transfer of the process from the surface determination unit to the component determination unit, the transfer of the process from the component determination unit to the state determination unit, and the transfer of the process from the state determination unit to the estimate calculation unit. One is the vehicle condition determination device according to any one of claims 1 to 5, wherein the determination is performed on condition that the user approves the determination result after allowing the user to modify the determination result. In the vehicle condition determination program that determines the damaged condition of the vehicle from the captured image,
Using the object detection algorithm by deep learning, the parts determination step to determine the vehicle parts in the captured image by referring to the parts learning model constructed by supervised learning using the data related to the vehicle parts as the teacher data, and
It has a state determination step of determining the damage state of each vehicle part determined by the parts determination unit with reference to a state learning model constructed by supervised learning using data on the damage state of vehicle parts as teacher data. Let the computer do the work
The object detection algorithm
By inputting the captured image into a single neural network system, the vehicle state determination is characterized in that the region extraction of the vehicle parts in the captured image is collectively performed with the classification of attributes by a regression problem approach. program. The vehicle state determination program according to claim 7, wherein the object detection algorithm is YOLO or SSD. It further has a surface determination step for determining the constituent surface of the vehicle in the captured image by referring to the surface learning model constructed by supervised learning using the data related to the constituent surface of the vehicle as the teacher data.
The seventh aspect of claim 7, wherein the component determination step filters the vehicle parts based on the constituent surface determined by the surface determination step, and outputs the filtered vehicle parts as a determination result. Vehicle condition determination program. The surface determination step according to claim 9, wherein when a vehicle component unique to a specific constituent surface is present in the captured image, the constituent surface corresponding to the specific constituent surface is included in the determination result. Vehicle condition determination program. By referring to the estimation table in which the repair cost associated with the damaged state of the vehicle part is held for each vehicle part, the repair cost of the vehicle is estimated from the damaged state of each vehicle part determined by the state determination step. The vehicle condition determination program according to any one of claims 7 to 10, further comprising an estimate calculation step. At least one of the transition of the process from the surface determination step to the component determination step, the transition of the process from the component determination step to the state determination step, and the transition of the process from the state determination step to the estimate calculation step. One is the vehicle condition determination program according to any one of claims 7 to 11, wherein the determination is performed on condition that the user's approval is obtained while allowing the user to modify the determination result. In the vehicle condition determination method for determining the damaged condition of the vehicle from the captured image,
Using the object detection algorithm by deep learning, the parts determination step to determine the vehicle parts in the captured image by referring to the parts learning model constructed by supervised learning using the data related to the vehicle parts as the teacher data, and
With reference to a state learning model constructed by supervised learning using data related to the damaged state of vehicle parts as teacher data, there is a state determination step for determining the damaged state of each vehicle part determined by the parts determination unit. death,
The object detection algorithm
By inputting the captured image into a single neural network system, the vehicle state determination is characterized in that the region extraction of the vehicle parts in the captured image is collectively performed with the classification of attributes by a regression problem approach. Method. The vehicle state determination method according to claim 13, wherein the object detection algorithm by deep learning is YOLO or SSD. It further has a surface determination step for determining the constituent surface of the vehicle in the captured image by referring to the surface learning model constructed by supervised learning using the data related to the constituent surface of the vehicle as the teacher data.
13. The thirteenth aspect of the present invention, wherein the component determination step filters the vehicle parts based on the constituent surface determined by the surface determination step, and outputs the filtered vehicle parts as a determination result. Vehicle condition determination method. The surface determination step according to claim 15, wherein when a vehicle component unique to a specific constituent surface is present in the captured image, the constituent surface corresponding to the specific constituent surface is included in the determination result. Vehicle condition determination method. By referring to the estimation table in which the repair cost associated with the damaged state of the vehicle part is held for each vehicle part, the repair cost of the vehicle is estimated from the damaged state of each vehicle part determined by the state determination step. The vehicle condition determination method according to any one of claims 13 to 16, further comprising an estimate calculation step. At least one of the transition of the process from the surface determination step to the component determination step, the transition of the process from the component determination step to the state determination step, and the transition of the process from the state determination step to the estimate calculation step. One is the vehicle condition determination method according to any one of claims 13 to 17, wherein the determination is performed on condition that the user's approval is obtained while allowing the user to modify the determination result.

Images