JP7036079B2 - Performance estimator - Google Patents

Description

The present invention relates to a performance estimation device that estimates the performance of an object.

In recent years, it has been proposed to perform classification and inspection of objects by using a convolutional neural network (for example, Patent Document 1). In the apparatus described in Patent Document 1, information on the performance (specifically, defects) of the object is acquired from the convolutional neural network trained by the teacher data using the image of the object as input data. ..

Japanese Unexamined Patent Publication No. 2018-5339

When estimating the performance of an object using a convolutional neural network, if the object has a three-dimensional shape (for example, the outer panel of a vehicle), the following inconveniences will occur. Even if an image of an object is simply prepared as input data for a convolutional neural network, the outer shape of the object can be grasped based on the image because the image is two-dimensional data, but the object is the same. It is not possible to grasp the uneven shape in the depth direction of. Therefore, the accuracy of estimating the performance of the object is low.

The present invention has been made in view of such circumstances, and an object of the present invention is to provide a performance estimation device capable of accurately estimating the performance of an object having a three-dimensional shape.

The performance estimation device for solving the above problems is a performance estimation device that estimates the performance of an object having a three-dimensional shape, and includes an input unit that stores a plurality of images of the same object at different viewing angles. A storage unit that stores a convolutional neural network learned by teacher data, which is a combination of a plurality of images of the same object having different viewing angles and information related to the performance of the same object, and a storage unit stored in the input unit. It has an output unit that acquires information related to the performance of the object from a learned convolutional neural network stored in the storage unit using the plurality of images as input data.

When having a plurality of images viewed from different directions, it is possible to accurately predict the shape of a three-dimensional object as compared with the case of having only one image viewed from one direction. .. According to the above configuration, a plurality of images viewed from such different directions can be used as input data of the convolutional neural network. Therefore, it is possible to accurately estimate the performance of an object having a three-dimensional shape.

In the performance estimation device, the teacher data has a plurality of data sets in which the plurality of images and the information related to the performance are associated with each other in a paired manner.
According to the above configuration, a multi-input convolutional neural network is created by training a convolutional neural network using "multiple images" as input data and "information related to the performance of an object" as output data for each data set. Can be built.

In the performance estimation device, the plurality of images are images showing a cross section of the object in different directions.
When having a plurality of images showing cross sections in different directions, it is possible to accurately grasp the shape of a three-dimensional object as compared with the case where there is only one image showing a cross section in one direction. become. According to the above configuration, since a plurality of images showing cross sections in different directions are used as input data of the convolutional neural network, the performance of the object having a three-dimensional shape can be estimated accurately based on the images. ..

According to the present invention, the performance of an object having a three-dimensional shape can be estimated with high accuracy.

The block diagram which shows the hardware composition of the performance estimation apparatus of one Embodiment. The schematic which shows the structure of the learning device of a performance estimation device. Top view of the hood of the vehicle to be estimated. A schematic diagram showing an example of a file name of teacher data. A flowchart showing the execution procedure of the learning process. A flowchart showing the execution procedure of the estimation process.

Hereinafter, an embodiment of the performance estimation device will be described.
The performance estimation device of the present embodiment is a device that estimates the pedestrian protection performance of the vehicle hood from the trained convolutional neural network by using an image showing the cross-sectional shape of the vehicle hood as input data. Specifically, the performance estimation device obtains the following (value A) to (value D) as information related to the pedestrian protection performance (specifically, the head protection performance) of the vehicle. In the present embodiment, the hood of the vehicle corresponds to an object having a three-dimensional shape, and the pedestrian protection performance corresponds to the performance of the hood.

[Value A] Head injury reference value (hereinafter referred to as HIC).
[Value B] A value corresponding to the amount of penetration of the head impactor for the pedestrian protection test into the vehicle hood (hereinafter referred to as the amount of impactor penetration).

[Value C] A value corresponding to the average acceleration of the head impactor at the initial stage of collision with the vehicle hood (1 ms) (hereinafter, 1 ms G).
[Value D] A value corresponding to the average acceleration of the head impactor at the initial stage of collision with the vehicle hood (2 ms) (hereinafter, 2 ms G).

Hereinafter, the hardware configuration of the performance estimation device of this embodiment will be described.
As shown in FIG. 1, the performance estimation device 10 is a computer to which a control unit 11, a storage unit 12, an input device 13, and an output device 14 are electrically connected.

The control unit 11 has a CPU 15, a RAM 16, a ROM 17, and the like, which are hardware processors. The control unit 11 is configured to execute various arithmetic processes based on various programs and data.

The storage unit 12 is composed of a hard disk drive, a solid state drive, or the like. The storage unit 12 includes an execution program 18 executed by the control unit 11, teacher data 19 used for machine learning of the learning device, learning data 20 created through the execution of machine learning, and an estimation target of pedestrian protection performance. The input unit 21 for storing the image of the hood, the estimation result data 22 indicating the result of estimating the pedestrian protection performance, and the like are stored.

The execution program 18 is a program for causing the performance estimation device 10 to execute the learning process and to generate the learning data 20 which is data indicating the execution result of the learning process, and the performance estimation device 10 executes the estimation process. It is a program for generating estimation result data 22 indicating the execution result of the estimation process. The teacher data 19 is data for performing machine learning of the learner through the learning process so as to acquire the ability to estimate the pedestrian protection performance. The details of the learning process, the estimation process, and the teacher data 19 will be described in detail later.

The input device 13 is a device for inputting a mouse, a keyboard, or the like. Further, the output device 14 is a device for outputting a display, a speaker, or the like. The performance estimation device 10 is operated by the input device 13 and the output device 14.

As shown in FIG. 2, the performance estimation device 10 uses a convolutional neural network (hereinafter referred to as CNN30) as a learned learner that has undergone machine learning for estimating pedestrian protection performance.

The input unit 21 of the storage unit 12 has a first input folder 21A and a second input folder 21B for storing image data. An image showing a front view cross section of the outer surface of the hood of the vehicle is stored in the first input folder 21A. An image showing a side view cross section of the outer surface of the hood of the vehicle is stored in the second input folder 21B. In the present embodiment, as shown by a broken line in a part of the cross-sectional line in FIG. 3, a plurality of places arranged in the front-rear direction of the vehicle (L direction in FIG. 3) as places for forming an image showing the front view cross section of the hood 25. (For example, 4000 places) are defined. Further, as locations for forming an image showing a side view cross section of the hood 25, a plurality of locations (4000 locations) arranged in the vehicle width direction (W direction in FIG. 3) are defined. Based on the images stored in the first input folder 21A (FIG. 2) and the second input folder 21B, the control unit 11 performs a process for estimating the pedestrian protection performance (learning process and estimation process) through the execution program 18. ) Is executed. In this embodiment, the image showing the front view cross section and the image showing the side view cross section correspond to a plurality of images of the same hood at different viewing angles, and correspond to images showing the cross sections of the same hood in different directions. do.

The performance estimation device 10 has two digitizing units 31A and 31B for digitizing image data. The first digitizing unit 31A digitizes the image data stored in the first input folder 21A, and the second digitizing unit 31B digitizes the image data stored in the second input folder 21B. In these digitization units 31A and 31B, the image data is converted into the value of the two-dimensional array.

The CNN 30 has a first input layer 32A and a second input layer 32B. The first input layer 32A and the second input layer 32B are layers in which numerical values of image data are input. The value of the two-dimensional array digitized by the first digitizing unit 31A is input to the first input layer 32A, and the value of the two-dimensional array digitized by the second digitizing unit 31B is input to the second input layer 32B. Is entered.

The CNN 30 has a first intermediate layer 33A and a second intermediate layer 33B. The first intermediate layer 33A and the second intermediate layer 33B each have a convolution layer 34 that performs a convolution process on the input data and a pooling layer 35 that performs a pooling process on the data output from the convolution layer 34. Have. The first intermediate layer 33A and the second intermediate layer 33B have a structure in which the convolutional layer 34 and the pooling layer 35 are alternately arranged a plurality of times (for example, three times).

CNN30 has a binding layer 36. The bonding layer 36 combines the value output from the first intermediate layer 33A and the value output from the second intermediate layer 33B.
CNN30 has four fully connected layers 37A, 37B, 37C, 37D. In each of these fully connected layers 37A to 37D, the data from which the characteristic portion is taken out through the first intermediate layer 33A and the second intermediate layer 33B is bound to one node, and the value converted by the activation function (characteristic variable). Is output. In the present embodiment, the first fully connected layer 37A is a layer that outputs a value corresponding to "HIC", and the second fully connected layer 37B is a layer that outputs a value corresponding to "impactor intrusion amount". The 3 fully connected layer 37C is a layer that outputs a value corresponding to "1 millisecond G", and the 4th fully connected layer 37D is a layer that outputs a value corresponding to "2 milliseconds G".

The CNN 30 has four output layers 38A, 38B, 38C, 38D. These output layers 38A to 38D are layers arranged on the most output side of the CNN 30. In the present embodiment, the first output layer 38A is a layer that calculates and outputs "HIC" based on the value output from the first fully connected layer 37A, and the second output layer 38B is the second fully connected layer. This layer calculates and outputs the "impactor intrusion amount" based on the value output from 37B. Further, the third output layer 38C is a layer that calculates and outputs "1 millisecond G" based on the value output from the third fully connected layer 37C, and the fourth output layer 38D is the fourth fully connected layer. It is a layer that calculates and outputs "2 milliseconds G" based on the value output from 37D. In this embodiment, these output layers 38A to 38D correspond to the output unit.

Hereinafter, the execution procedure of the learning process will be described.
When executing the learning process, the control unit 11 expands the execution program 18 stored in the storage unit 12 into the RAM 16. Then, the control unit 11 executes the execution program 18 (specifically, the learning process) by the CPU 15 to execute the machine learning of the CNN 30.

Further, when the learning process is executed, the image (specifically, the teacher data 19) is stored in the first input folder 21A and the second input folder 21B of the storage unit 12. Machine learning of CNN30 is executed based on this image.

The teacher data 19 is created based on the result of actually measuring the pedestrian protection performance of the developed vehicle. The teacher data 19 includes an image showing a front view cross section of the outer surface of the hood, an image showing a side view cross section of the outer surface of the hood, and each value of pedestrian protection performance (HIC, impactor penetration amount, 1 ms G, 2 mm). A lot of data combined with the second G) are prepared.

The teacher data 19 is a data set in which an image showing a front view cross section, an image showing a side view cross section, and each value of pedestrian protection performance are associated with each other by the file name of the image saved in each input folder 21A and 21B. It has become. Specifically, among the large number of images stored in the first input folder 21A and the second input folder 21B, the pair of images to be combined are given the same file name. In addition, this file name contains information about each value of the pedestrian protection performance when these images are combined. By extracting images with the same file name from the input folders 21A and 21B separately when executing the learning process, an image showing a front view cross section, an image showing a side view cross section, and each value of pedestrian protection performance are set. You can get the data that became.

An example of such a file name is shown in FIG. In the example shown in FIG. 4, the file name "L1000" indicates the position in the L direction of the front view cross section, and the file name "W200" indicates the position in the W direction of the side view cross section. Specifically, the file name "L1000W200" is data obtained by combining the 1000th cross section from the front side of the vehicle in the front view cross section and the 200th cross section from the left side of the vehicle in the side view cross section. Shows. Further, "HIC500" in the above file name indicates that the HIC at the intersection (dotted point coordinates) between the front view cross section and the side view cross section is "500", and "ST90" in the same file name is in the hitting point coordinates. It shows that the impactor intrusion amount is "90". Furthermore, the above file name "1G"
"20" indicates that the 1 millisecond G in the dot coordinates is "20", and "2G10" in the same file name indicates that the 2 milliseconds G in the dot coordinates is "10". .. In the present embodiment, as the teacher data 19, a large number of sets (for example, several thousand sets) of data are prepared for each of the dot coordinates.

As shown in FIG. 5, in the performance estimation device 10 of the present embodiment, an image showing a front view cross section of the hood in the teacher data 19 is stored in the first input folder 21A in accordance with the execution of the learning process, and the same. An image showing a side view cross section of the hood is stored in the second input folder 21B (step S11).

Then, in the learning process, machine learning of the CNN 30 as a learning device is executed based on the images stored in the input folders 21A and 21B (step S12). Specifically, based on the file names of the images stored in the input folders 21A and 21B, a set of image data in which the image showing the front view cross section and the image showing the side view cross section are paired is extracted. Then, when those images are used as the input data of the CNN 30, the CNN 30 is constructed so that each value of the pedestrian protection performance included in the file name of the image is obtained as the output value of the CNN 30. In the learning process of the present embodiment, such machine learning of the CNN 30 is repeatedly executed in such a manner that such machine learning is executed for each set of image data in which an image showing a front view cross section and an image showing a side view cross section are paired. Then, the configuration of the CNN30 constructed at this time (for example, the number of neurons in each layer, the connection relationship between neurons, the transmission function of each neuron), the weight of the connection between each neuron, and the information indicating the threshold value of each neuron are learned. It is stored in the storage unit 12 as data 20. In the present embodiment, the execution program 18 is constructed and stored in the storage unit 12 so that such a series of processes are automatically executed.

Hereinafter, the execution procedure of the estimation process will be described.
When executing the estimation process, the control unit 11 expands the execution program 18 stored in the storage unit 12 into the RAM 16. Information indicating the structure of the learned CNN 30, the weight of the connection between each neuron, and the threshold value of each neuron is included in the learning data 20 stored in the storage unit 12. In the present embodiment, when the execution program 18 is developed, such learning data 20 is referred to, and the trained CNN 30 used for the estimation process for estimating the pedestrian protection performance is set.

As shown in FIG. 6, in the performance estimation device 10 of the present embodiment, an image showing a front view cross section of the hood 25 of the estimation target vehicle through the operation of the input device 13 is displayed in the first input folder 21A in accordance with the execution of the estimation process. An image showing a side view cross section of the hood 25 is stored in the second input folder 21B (step S21).

In the present embodiment, when the design shape is determined in the development process of the estimation target vehicle, an image showing a side view cross section of the hood 25 (specifically, its outer surface) and a front view based on the design shape are obtained. An image showing a cross section is formed. Then, these images are saved in the first input folder 21A and the second input folder 21B in accordance with the execution of the estimation process.

As such an image for estimation, a pair of image data in which an image showing a front view cross section of the hood 25 and an image showing a side view cross section of the hood 25 are combined is prepared for each dot coordinate. The image data for estimation is a data set in which an image showing a front view cross section and an image showing a side view cross section are associated with each other by the file name given to the image saved in each input folder 21A and 21B. .. Specifically, among a large number of image data stored in the first input folder 21A and the second input folder 21B, the same file name (for example, L1000W200.jpg) is given to the pair of image data to be combined. Has been done. Therefore, when executing the estimation process, by extracting images with the same file name from the input folders 21A and 21B, data obtained as a set of an image showing a front view cross section and an image showing a side view cross section can be obtained. Can be done. In the illustrated file name (L1000W200.jpeg), "L1000" indicates the position in the L direction of the front view cross section, and "W200" in the same file name indicates the position in the W direction of the side view cross section. Therefore, this file name indicates that the data is a combination of the 1000th cross section from the front side of the vehicle in the front view cross section and the 200th cross section from the right side of the vehicle in the side view cross section.

In the estimation process of the present embodiment, each output value of the pedestrian protection performance at the intersection (dotting coordinates) between the front view cross section and the side view cross section is used based on the pair of image data and the trained CNN30 is used. (HIC, impactor penetration amount, 1 ms G, 2 ms G) is required (step S22). In the present embodiment, the calculation of each output value for the pedestrian protection performance is repeatedly executed in such a manner that the calculation of each output value is executed for each data set in which the image showing the front view cross section and the image showing the side view cross section are paired. To. In the present embodiment, the execution program 18 is constructed and stored in the storage unit 12 so that such a series of processes are automatically executed.

Then, when the calculation of the output value of the pedestrian protection performance for all the data sets is completed, those output values are output (step S23). Specifically, data (for example, CSV data) showing the relationship between each impact hit point and each output value for pedestrian protection performance is created, and the data is stored in the storage unit 12 as estimation result data 22. To. Further, a table showing the relationship between the impact hitting point and each output value for the pedestrian protection performance is created and displayed on the output device 14 (specifically, the display).

According to this embodiment, the effects described below can be obtained.
(1) As the input data of the CNN 30 and the image data constituting the teacher data 19, two types of images having different viewing angles, specifically, an image showing a front view cross section and an image showing a side view cross section of the hood are prepared. Here, when predicting the shape of a three-dimensional hood, a cross section in a different direction is shown as compared with a case where only one image showing a cross section in one direction is prepared as an image used for the prediction. It is possible to predict the shape of the hood more accurately when two images are prepared. According to this embodiment, two images showing such cross sections in different directions can be used as input data and teacher data 19 of CNN 30. Therefore, it is possible to accurately estimate the pedestrian protection performance of the hood 25 while accurately grasping and considering the shape of the hood 25 having a three-dimensional shape.

(2) The teacher data 19 has a plurality of data sets in which two images showing a cross section of the hood and each value of the pedestrian protection performance by the hood are associated with each other in a paired manner. As a result, the CNN30 is trained using "two images showing the cross section of the hood" as input data and "each value of the pedestrian protection performance by the hood" as output data for each data set, so that the CNN30 has multiple inputs. Can be built.

The above embodiment may be modified as follows.
-The configuration of CNN30 can be changed arbitrarily. For example, the number of times the convolutional layer 34 and the pooling layer 35 are alternately arranged in the intermediate layers 33A and 33B may be changed, or a dropout layer may be added.

-The item to be estimated by the performance estimation device 10 is set to one of HIC, impactor intrusion amount, 1 ms G, and 2 ms G, any two, or any three. May be good. Further, values other than the HIC, the impactor intrusion amount, 1 millisecond G, and 2 ms G can be set as the estimation target items of the performance estimation device 10.

-The mode for outputting the estimation result of the pedestrian protection performance is not limited to storing the data indicating the estimation result in the storage unit 12 or displaying the data in the output device 14, and can be arbitrarily changed. ..

-Information indicating the thickness of the hood and information indicating the material of the hood may be added to the input data of the CNN 30 and the image data constituting the teacher data 19. For example, information indicating the thickness of the hood can be given by changing the thickness of the line indicating the cross section of the hood, and information indicating the material of the hood can be given by changing the display color of the line indicating the cross section of the hood. Can be granted. According to the above configuration, since each value of the pedestrian protection performance can be calculated after grasping the thickness and material of the hood, those values can be calculated more accurately.

-The teacher data 19 may be data in which an image showing a front view cross section of the hood, an image showing a side view cross section of the hood, and a numerical data file showing each value of the pedestrian protection performance are combined. In short, any data set in which the image showing the front view cross section of the hood, the image showing the side view cross section of the hood, and each value of the pedestrian protection performance are associated can be used as the teacher data 19.

The image data constituting the input data of the CNN 30 and the teacher data 19 is not limited to an image showing a front view cross section and an image showing a side view cross section of the hood, but an image showing a cross section of the hood along an arbitrary cross section. Can be used. Further, as the image data constituting the input data of the CNN 30 and the teacher data 19, not only the image showing the cross section of the hood but also the image showing the outer surface shape of the hood can be used.

-Three or more images may be used as the image data constituting the input data of the CNN 30 and the teacher data 19. In this case, an input folder for storing the image data, a digitizing unit for quantifying the image data, an input layer for CNN, and an intermediate layer may be set in a manner corresponding to each of the image data. good. For example, an image showing a plurality of front view cross sections having different positions in the L direction and an image showing a plurality of side view cross sections having different positions in the W direction are used as image data constituting the input data of the CNN 30 and the teacher data 19. Can be done. According to such a configuration, it is possible to estimate the pedestrian protection performance by the hood after considering the shape around the hitting point coordinates in the hood. In addition, as image data constituting the input data of the CNN 30 and the teacher data 19, an image showing a front view cross section of the hood, an image showing a side view cross section, and an image showing a cross section other than the front view cross section and the side view cross section. It is also possible to use.

-The CNN used for estimating the pedestrian protection performance may be set for each vehicle type. If the vehicle is the same model, the basic structure of the hood will not change significantly when the vehicle is remodeled. Therefore, as in the performance estimation device of the above embodiment, each value of the pedestrian protection performance is obtained from the trained CNN 30 using an image showing the cross-sectional shape of the outer surface of the hood as input data, and these values are obtained accurately. Can be done. According to the above configuration, the pedestrian protection performance by the hood can be accurately estimated by using the CNN set for each vehicle type.

-The performance estimation device according to the above embodiment is not limited to the device that estimates the pedestrian head protection performance by the vehicle hood, but also applies to the device that estimates the pedestrian leg protection performance by the vehicle bumper. Can be done. In this case, the leg protection performance may be estimated from the trained CNN using an image showing the design shape of the bumper as input data. The performance estimation device according to the above embodiment can be applied as long as it is a device that estimates the performance of the constituent members having different viewing angles based on a plurality of images of the constituent members constituting the vehicle. Further, the device is not limited to a device that estimates the performance of the constituent members constituting such a vehicle, but may be a device that estimates the performance of the object having a three-dimensional shape based on a plurality of images having different viewing angles. For example, the performance estimation device according to the above embodiment can be applied.

10 ... Performance estimation device, 11 ... Control unit, 12 ... Storage unit, 13 ... Input device, 14 ... Output device, 15 ... CPU, 16 ... RAM, 17 ... ROM, 18 ... Execution program, 19 ... Teacher data, 20 ... Learning data, 21 ... input unit, 21A ... first input folder, 21B ... second input folder, 22 ... estimation result data, 25 ... hood, 30 ... convolutional neural network (CNN), 31A ... first digitization unit, 31B ... 2nd digitization unit, 32A ... 1st input layer, 32B ... 2nd input layer, 33A ... 1st intermediate layer, 33B ... 2nd intermediate layer, 34 ... folding layer, 35 ... pooling layer, 36 ... bonding layer, 37A ... 1st total bond layer, 37B ... 2nd total bond layer, 37C ... 3rd total bond layer, 37D ... 4th total bond layer, 38A ... 1st output layer, 38B ... 2nd output layer, 38C ... 3rd Output layer, 38D ... 4th output layer.

Claims (3)

In a performance estimation device that estimates the performance of an object that forms a three-dimensional shape,
An input unit that stores a plurality of images of the same object at different viewing angles,
A storage unit that stores a convolutional neural network learned by teacher data, which is a combination of a plurality of images of the same object having different viewing angles and information related to the performance of the same object.
An output unit that acquires information related to the performance of the object from the learned convolutional neural network stored in the storage unit using the plurality of images stored in the input unit as input data.
A performance estimation device characterized by having. The performance estimation device according to claim 1, wherein the teacher data has a plurality of data sets in which the plurality of images and the information related to the performance are associated with each other in a paired manner. The performance estimation device according to claim 1 or 2, wherein the plurality of images are images showing a cross section of the object in a different direction.

Images