JP7349626B2 - Model generation device, vehicle simulation system, model generation method, vehicle simulation method, and computer program - Google Patents

Description

The present disclosure relates to data processing technology, and particularly to a model generation device, a vehicle simulation system, a model generation method, a vehicle simulation method, and a computer program.

In the development and evaluation phases of in-vehicle equipment, in order to reduce the cost required for development and evaluation, a vehicle simulation system that simulates the behavior of a real vehicle is sometimes used instead of using a real vehicle.

Special table 2018-514042 publication

When simulating the behavior of a vehicle when it detects an object such as a pedestrian, the simulation results may not be sufficiently useful unless the ability to detect the object is based on reality. There is.

The present disclosure has been made in view of these circumstances, and one purpose is to increase the usefulness of simulation results in a vehicle simulation system.

In order to solve the above problems, a model generation device according to an aspect of the present disclosure includes a first detection section that detects the state of the surroundings of the vehicle from image data showing the surroundings of the vehicle; This is a model for estimating the detection rate by the first detection unit in a vehicle simulation system, including a second detection unit that detects a surrounding state, the state detected by the second detection unit being an explanatory variable, and the second detection unit detecting a surrounding state. The generating unit includes a generating unit that generates a model whose objective variable is a probability that the state detected by the detecting unit is detected by the first detecting unit.

Another aspect of the present disclosure is a vehicle simulation system. This vehicle simulation system is a model for estimating the detection rate of the detection unit that detects the conditions around the vehicle from image data showing the surroundings of the vehicle, and uses the true value of the condition around the vehicle as an explanatory variable. By inputting the true value of the state around the vehicle as a simulation parameter into the model, a storage unit stores a model whose objective variable is the probability that the state around the vehicle is detected by the detection unit. The vehicle includes an estimator that estimates a detection rate, and a simulation unit that simulates the behavior of the vehicle in accordance with the detection rate estimated by the estimator.

Yet another aspect of the present disclosure is a model generation method. In this method, the first detection section detects the state around the vehicle from image data showing the surroundings of the vehicle, and the second detection section detects the state around the vehicle with higher accuracy than the first detection section. This is a model for estimating the detection rate by the first detection unit in a simulation system, in which the state detected by the second detection unit is an explanatory variable, and the state detected by the second detection unit is Generate a model whose objective variable is the probability of being detected by .

Yet another aspect of the present disclosure is a vehicle simulation method. This method is a model for estimating the detection rate of a detection unit that detects the state of the surroundings of a vehicle from image data showing the surroundings of the vehicle, and uses the true value of the state of the surroundings of the vehicle as an explanatory variable. By storing a model whose objective variable is the probability that the state around the vehicle will be detected by the detection unit, and inputting the true value of the state around the vehicle as a simulation parameter into the model, the detection rate by the detection unit is estimated. , the behavior of the vehicle is simulated based on the estimated detection rate.

Note that any combination of the above components and expressions of the present disclosure converted between devices, systems, computer programs, recording media on which computer programs are recorded, etc. are also effective as aspects of the present disclosure.

According to the present disclosure, the usefulness of simulation results in a vehicle simulation system can be increased.

FIG. 2 is a block diagram showing functional blocks of a conventional vehicle simulation system. FIG. 3 is a diagram showing an example of sample data for constructing a sensor model. FIG. 2 is a block diagram showing functional blocks of the sensor model generation device of the first embodiment. FIG. 1 is a block diagram showing functional blocks of a vehicle simulation system according to a first embodiment. 5 is a block diagram showing details of the sensor model section of FIG. 4. FIG. It is a figure which shows the example of curve fitting. FIG. 7 is a diagram showing the state around the vehicle in a modified example. FIG. 3 is a diagram showing an example of sample data for constructing a distance error model. FIG. 3 is a diagram showing an example of sample data for constructing a detection rate model. FIG. 7 is a diagram showing an example of a detection rate table for a certain posture. FIG. 7 is a block diagram showing functional blocks of a simulation control unit in a second embodiment. It is a figure which shows the example of a simulation result image. It is a figure which shows the example of a simulation result image. It is a figure which shows the example of a simulation result image. FIG. 3 is a diagram showing an example of sample data for constructing a detection rate model. FIG. 3 is a block diagram showing functional blocks of a sensor model generation device according to a third embodiment. FIG. 7 is a block diagram showing functional blocks of a sensor model section of a third embodiment. FIG. 7 is a block diagram showing functional blocks of a sensor model section of a modified example.

<First example>
FIG. 1 is a block diagram showing functional blocks of a conventional vehicle simulation system 100. Each block shown in the block diagram of the present disclosure can be realized in terms of hardware by elements and mechanical devices such as the CPU and memory of a computer, and in terms of software by a computer program, etc.; , depicts the functional blocks realized by their cooperation. Those skilled in the art will understand that these functional blocks can be implemented in various ways by combining hardware and software.

The vehicle simulation system 100 includes a simulation control section 10, a user interface 12, an environmental data generation section 14, an image analysis section 16, and a vehicle model section 18. The simulation control unit 10 controls simulation of vehicle behavior. The simulation control unit 10 also transmits and receives data to and from the user interface 12, the environmental data generation unit 14, the image analysis unit 16, and the vehicle model unit 18.

The user interface 12 provides an interface with the user (or the user's terminal). The environmental data generation section 14 generates image data (CG) to be analyzed by an image analysis section 16, which will be described later, according to parameters specified by the user (for example, distance to a pedestrian, direction, etc.). For example, the environmental data generation unit 14 generates a CG indicating that there is a pedestrian 6 meters ahead of the own vehicle.

The image analysis unit 16 analyzes image data to detect conditions around the vehicle (positions of obstacles, etc.) for controlling automatic travel of the vehicle. For example, the image analysis unit 16 analyzes a CG indicating that there is a pedestrian 6 meters ahead of the vehicle in the direction of travel, and detects that the distance between the vehicle and the pedestrian is 6.2 meters. In this case, there is an error of 0.2 meters. Note that the object to be analyzed by the image analysis unit 16 is an image captured by an on-vehicle camera in an actual vehicle, but in the vehicle simulation system 100, it is a CG generated by the environmental data generation unit 14.

The vehicle model unit 18 determines the behavior of the vehicle based on the analysis result by the image analysis unit 16, that is, simulates automatic driving of the vehicle. For example, if the image analysis unit 16 detects that there is a pedestrian 6.2 meters ahead in the direction of travel of the own vehicle, the vehicle model unit 18 may operate the brakes and, as a result, It may also output that the vehicle will stop in front of you. The simulation control unit 10 displays the simulation results by the vehicle model unit 18 on a predetermined display device or stores them in a predetermined storage device.

As described above, in the conventional vehicle simulation system 100, an image showing the state around the vehicle is created, the state around the vehicle is detected from the image, and the content of automatic driving is simulated. In order to increase the comprehensiveness of the simulation, it is necessary to create many images that correspond to various cases such as the brightness around the vehicle (weather, etc.), road surface conditions, distance to obstacles, vehicle speed, etc. Creating many images requires a lot of time and money.

It is also possible to create an image showing the state around the vehicle based on a theoretical model (for example, a theoretical calculation formula based on multiple parameters such as vehicle speed and brightness); Images may deviate from the real environment. As a result, the results of the vehicle simulation may deviate from the actual behavior of the vehicle.

Therefore, in the first embodiment, based on the environmental data indicating the actual state around the vehicle (in other words, the data of the real environment) and the image analysis results, a mathematical model (hereinafter referred to as (also called "sensor model"). The sensor model is then used to simulate the behavior of the vehicle. This eliminates the need to create images for each case in a simulation that detects the state around the vehicle, which reduces simulation costs and allows simulation of vehicle behavior that is consistent with reality.

First, the sensor model of the first embodiment will be explained. The sensor model of the first embodiment is a model for estimating detection results based on images (that is, detection results by the image analysis unit 16 and the image analysis unit 28 described below) in a vehicle simulation system. FIG. 2 shows an example of sample data for building a sensor model. The figure shows sample data for constructing a sensor model related to pedestrian identification. Each sample includes distance from the vehicle to the pedestrian and weather as two factors that affect pedestrian identification accuracy.

The factor "distance" includes 10 categories corresponding to multiple levels of distance (true value) from the vehicle to the pedestrian. For example, one category may range from 0.1 meters to 2 meters, and another category may range from 2 meters to 4 meters. The factor "weather" includes three categories: sunny, cloudy, and rainy. For each sample, the value of the applicable category is "1", and the value of the non-applicable category is "0". This allows not only quantitative data but also qualitative data (in other words, data regarding quality) to be treated as quantity. Note that factors are also called explanatory variables. Each category of factors is also called a dummy variable.

In addition, each sample is used as data indicating the accuracy of image analysis, which is the difference between the true value of the distance from the vehicle to the pedestrian and the distance from the vehicle to the pedestrian detected by image analysis (also known as "detection distance error"). ). The sensor model of the embodiment includes a plurality of dummy variables corresponding to multiple stages of distance from the vehicle to a detection target (such as a pedestrian). Specifically, by using a total of 13 items (10 categories of distance and 3 categories of weather) as dummy variables and performing multiple regression analysis with detection distance error as the objective variable, we calculated the influence of each category of each factor on detection distance error. Generate a sensor model that shows the size of.

式１のε_ｋは残差であり、ａ^ｉｊはｉ番目の要因のｊ番目のカテゴリを表すダミー変数にかかる係数である（以下「カテゴリ係数」とも呼ぶ。）。実施例のセンサモデル生成装置は、重回帰分析によって、残差の平方和が最小になるように各ダミー変数にかかるカテゴリ係数を導出する。 Generalizing the number _of factors as m, the number of categories of each factor as n ₁ , .

In Equation 1, ε _k is a residual, and a ^ij is a coefficient applied to a dummy variable representing the j-th category of the i-th factor (hereinafter also referred to as "category coefficient"). The sensor model generation device of the embodiment derives category coefficients for each dummy variable by multiple regression analysis so that the sum of squares of residual errors is minimized.

FIG. 3 is a block diagram showing functional blocks of the sensor model generation device 20 of the first embodiment. The sensor model generation device 20 includes a sensor data storage section 22, a reference data storage section 24, a tag data storage section 26, an image analysis section 28, a true value detection section 30, an error derivation section 32, a true value quantization section 34, a tag quantum It includes a converting section 36 and a model generating section 38.

A computer program including a plurality of modules implementing the functions of the plurality of functional blocks shown in FIG. 3 may be stored in the storage of the sensor model generation device 20. The CPU of the sensor model generation device 20 may perform the functions of each functional block shown in FIG. 3 by reading this computer program into the main memory and executing it. Further, the plurality of functions shown in FIG. 3 may be distributed to a plurality of devices, or may be realized by the plurality of devices working together as a system.

The sensor data storage unit 22 and the reference data storage unit 24 store data collected in tests using actual vehicles. The sensor data storage unit 22 stores sensing results of various on-vehicle sensors regarding objects to be detected. In the embodiment, the sensor data storage unit 22 stores a plurality of image data captured by an on-vehicle camera, such as image data showing a pedestrian 3 meters away, and image data showing a pedestrian 6 meters away. It stores image data showing the state, etc.

The reference data storage unit 24 stores data for true value detection regarding the object to be detected. In the embodiment, the reference data storage unit 24 stores data collected by LIDAR (Light Detection and Ranging), including the true value of the distance between the vehicle and the object. The tag data storage unit 26 stores data set by a user (developer, tester, etc.) as tag data. For example, the tag data storage unit 26 stores tag data indicating the weather on the day when a test using a vehicle was conducted. The tag data may include various data such as the type of road surface (for example, asphalt or dirt), temperature, season, etc.

Note that the data stored in the sensor model generation device 20, the sensor data storage unit 22, and the reference data storage unit 24, which are collected or input in the same test, are associated with each other and become one set of sample data. . A plurality of sets of sample data (for example, 10,000 sets of sample data as shown in FIG. 2) are stored in the sensor model generation device 20, the sensor data storage section 22, and the reference data storage section 24.

The image analysis section 28 corresponds to the image analysis section 16 in FIG. That is, the image analysis unit 28 detects the state around the vehicle for controlling automatic driving of the vehicle based on the image data stored in the sensor data storage unit 22. Specifically, the image analysis unit 28 detects the distance between the vehicle and the pedestrian in a plurality of samples based on the image data of the plurality of samples stored in the sensor data storage unit 22. A known technique may be used to derive the distance between the vehicle and the pedestrian based on the image.

The true value detection unit 30 detects the state around the vehicle based on the LIDAR data stored in the reference data storage unit 24. Specifically, the true value detection unit 30 detects the distance between the vehicle and the pedestrian in a plurality of samples based on the LIDAR data of the plurality of samples stored in the reference data storage unit 24. Here, distance detection using LIDAR is more accurate than distance detection using images, and the result of distance detection using LIDAR essentially indicates the true value of the distance between the vehicle and the pedestrian. That is, it can be said that the true value detection unit 30 detects the true value of the distance between the vehicle and the pedestrian.

The error derivation unit 32 calculates, for each sample, the detection result by the true value detection unit 30 (in the example, the true value of the distance between the vehicle and the pedestrian) and the detection result by the image analysis unit 28 (in the example, the distance between the vehicle and the pedestrian using the image). The detected distance error is calculated as the difference between the detected distance between the vehicle and the pedestrian.

The true value quantization unit 34 converts the detection result by the true value detection unit 30 (in the embodiment, the true value of the distance between the vehicle and the pedestrian) into one of a plurality of categories of predetermined explanatory variables for each sample. Classify into. For example, as shown in FIG. 2, the true value quantization unit 34 classifies the true value of the distance between the vehicle and the pedestrian detected by the true value detection unit 30 into a corresponding category out of ten categories. You may.

The tag quantization unit 36 classifies the tag data stored in the tag data storage unit 26 into one of a plurality of predetermined explanatory variable categories for each sample. For example, as shown in FIG. 2, the tag quantization unit 36 may classify tag data indicating the weather into the appropriate category among three categories.

The model generation unit 38 creates a combination of the error calculated by the error derivation unit 32, the category selected by the true value quantization unit 34, and the category selected by the tag quantization unit 36 for each sample. The model generation unit 38 uses the error calculated by the error derivation unit 32 as an objective variable, and performs statistical processing using the category selected by the true value quantization unit 34 and the category selected by the tag quantization unit 36 as dummy variables. (Multiple regression analysis in the embodiment) is performed to derive a plurality of category coefficients related to a plurality of categories. The model generation unit 38 generates a sensor model using a regression equation (formula 1 above) in which category coefficients are set, and stores the coefficients of a plurality of categories obtained as data of the sensor model in the model storage unit 40.

Next, a vehicle simulation system that uses the sensor model generated by the sensor model generation device 20 will be described. FIG. 4 is a block diagram showing functional blocks of the vehicle simulation system 110 of the first embodiment. The vehicle simulation system 110 of the first embodiment includes a simulation control section 10, a user interface 12, an environmental data generation section 14, a vehicle model section 18, and a sensor model section 50.

A computer program including a plurality of modules implementing the functions of the plurality of functional blocks shown in FIG. 4 may be stored in the storage of the vehicle simulation system 110. The CPU of the vehicle simulation system 110 (or the CPU of a device within the system) may perform the functions of each functional block shown in FIG. 4 by reading this computer program into the main memory and executing it. Moreover, the plurality of functions shown in FIG. 4 may be distributed to a plurality of devices, or may be realized by the plurality of devices working together as a system. Furthermore, multiple functions shown in FIG. 4 may be aggregated into a single device.

Among the functional blocks of the vehicle simulation system 110 of the first embodiment, functional blocks that are the same as or correspond to the functional blocks of the conventional vehicle simulation system 100 shown in FIG. 1 are given the same reference numerals. Hereinafter, the content that has already been explained in connection with FIG. 1 will be omitted from further explanation as appropriate.

The environmental data generation unit 14 generates environmental data that is a prerequisite for simulation according to parameters accepted through the user interface 12 or parameters input from an external source such as a file. The environmental data includes values of explanatory variables of the sensor model. The environmental data of the embodiment includes the distance from the vehicle to the pedestrian as true value data, or includes data necessary for calculating the distance from the vehicle to the pedestrian, and includes a value indicating the weather as tag data.

The sensor model unit 50 provides a function that replaces the image analysis unit 16 in the conventional vehicle simulation system 100. The sensor model section 50 inputs the true value of the state around the vehicle (in the embodiment, the distance from the vehicle to the pedestrian) and tag data as vehicle simulation parameters, so that the image analysis section 16 can calculate the above state. It functions as an estimator that estimates the detection result when an image showing the image is analyzed.

The vehicle model unit 18 determines the behavior of the vehicle based on the estimation result by the sensor model unit 50, that is, simulates automatic driving of the vehicle. For example, if the sensor model unit 50 determines that there is a pedestrian 6.2 meters ahead of the vehicle in the direction of travel, the vehicle model unit 18 may operate the brakes, It may also output that the vehicle will stop a meter before the vehicle stops. The simulation control unit 10 may display the simulation results by the vehicle model unit 18 on a display device, or may store them in a storage device.

FIG. 5 is a block diagram showing details of the sensor model section 50 of FIG. 4. As shown in FIG. The sensor model section 50 includes a tag quantization section 52, a true value quantization section 54, an error derivation section 56, a simulated value derivation section 58, and a model storage section 40. The model storage unit 40 stores the sensor model generated by the sensor model generation device 20.

The tag quantization unit 52 receives the tag data generated by the environmental data generation unit 14 via the simulation control unit 10, and classifies it into one of a plurality of predetermined explanatory variable categories. For example, as shown in FIG. 2, the tag quantization unit 52 may classify tag data indicating weather into the appropriate category among three categories.

The true value quantization unit 54 receives the true value data generated by the environmental data generation unit 14 or the data for calculating the true value (hereinafter collectively referred to as “true value data”) via the simulation control unit 10. It is accepted and classified into one of a plurality of categories of predetermined explanatory variables. For example, as shown in FIG. 2, the true value quantization unit 54 may classify the true value of the distance from the vehicle to the pedestrian into a corresponding category among ten categories.

The error derivation unit 56 derives a detection distance error according to the true value data and tag data generated by the environmental data generation unit 14 and the sensor model (for example, Equation 1 above) stored in the model storage unit 40. . In the embodiment, the error derivation unit 56 sets the value of the dummy variable corresponding to the category selected by the tag quantization unit 52 and the true value quantization unit 54 to 1 (into an unselected category) among the dummy variables of the sensor model. By setting the corresponding dummy variable value to 0), the detection distance error, which is the target variable of the sensor model, is calculated using the coefficient of the category corresponding to the dummy variable that becomes 1.

The simulated value derivation unit 58 adds the detection distance error derived by the error derivation unit 56 to the true value data generated by the environmental data generation unit 14, thereby determining the detection result by the image analysis unit 16 (image analysis unit 28). to simulate. For example, the simulated value derivation unit 58 calculates the distance to the pedestrian that the image analysis unit 16 would detect if there was a pedestrian 6 meters ahead of the actual vehicle (a value that includes an error, for example, 6.2 m). The value derived by the simulated value deriving unit 58 (for example, the distance to a pedestrian) is input to the vehicle model unit 18 via the simulation control unit 10, and the behavior of the vehicle is simulated.

The operation of the above configuration will be explained.
First, the operation of the sensor model generation device 20 will be described with reference to FIG. 3. Developers of in-vehicle equipment who are supposed to provide vehicle simulation systems to customers and partner companies actually drive the vehicle, capture images of pedestrians outside the vehicle with the in-vehicle camera, and store multiple samples of captured images in the sensor data storage unit 22. Make me remember. At the same time, the developer causes the LIDAR device to measure the distance to the pedestrian, and causes the reference data storage unit 24 to store the measurement results indicating the true values of the distances of the plurality of samples. Furthermore, the developer causes the tag data storage unit 26 to store a plurality of samples of tag data indicating qualitative content such as the weather.

The image analysis unit 28 detects the distance to the pedestrian from the image stored in the sensor data storage unit 22 for each sample. The true value detection unit 30 detects the true value of the distance to the pedestrian from the measurement results stored in the reference data storage unit 24 for each sample. The error derivation unit 32 derives a detected distance error, which is the difference between the detected distance based on the image and the true value, for each sample. The true value quantization unit 34 specifies the category of the true value of the distance to the pedestrian, and the tag quantization unit 36 specifies the category of tag data.

The model generation unit 38 uses the category of the true value of the distance to the pedestrian and the category of the tag data as dummy variables, and uses the detection distance error as the objective variable, and executes multiple regression analysis using a plurality of samples. Generate the model. The model generation unit 38 stores the generated sensor model in the model storage unit 40.

According to the sensor model generation device 20 of the first embodiment, if detection results by image analysis are required when simulating vehicle behavior, images of each case to be simulated are prepared, and images of each case are Analysis becomes unnecessary, reducing the time and cost required for simulation. Furthermore, by generating a sensor model using data indicating the actual conditions around the vehicle, it is possible to generate a sensor model that outputs results consistent with reality.

Next, the operation of the vehicle simulation system 110 will be described with reference to FIGS. 4 and 5. Here, we will simulate the behavior of a vehicle when a pedestrian exists near the vehicle. The user inputs the distance from the vehicle to the pedestrian and the weather as simulation parameters to the user interface 12 of the vehicle simulation system 110. The environmental data generating unit 14 generates true value data indicating the distance from the vehicle to the pedestrian, which is accepted by the user interface 12, and also generates tag data indicating the weather, which is accepted by the user interface 12.

The tag quantization section 52 specifies the category of tag data (for example, the weather in FIG. 2), and the true value quantization section 54 specifies the category of the true value data (for example, the distance in FIG. 2). The error derivation unit 56 calculates the categories (dummy variables) specified by the tag quantization unit 52 and the true value quantization unit 54 among the dummy variables indicating the categories of the explanatory variables of the sensor model stored in the model storage unit 40. By setting the value to "1" and using the coefficient (ie, model data) corresponding to that category, the detection distance error, which is the target variable, is obtained. The simulated value derivation unit 58 estimates the distance (including error) from the vehicle to the pedestrian detected by the image analysis unit 28 by adding the detected distance error to the distance (true value) from the vehicle to the pedestrian. .

The vehicle model unit 18 determines the behavior of the vehicle based on the distance (including error) from the vehicle to the pedestrian derived by the simulated value derivation unit 58. The simulation control unit 10 outputs the content determined by the vehicle model unit 18 as a simulation result to a predetermined output device or stores it in a predetermined storage device.

According to the vehicle simulation system 110 of the first embodiment, if detection results by image analysis are required when simulating vehicle behavior, it is possible to prepare images for each case to be simulated and analyze the images for each case. This eliminates the need for simulation, reducing the time and cost required for simulation. Furthermore, it becomes easier to improve the comprehensiveness of the simulation. Furthermore, by using a sensor model generated using the actual conditions around the vehicle, it is possible to simulate the vehicle behavior in accordance with reality.

The present disclosure has been described above based on the first embodiment. The first embodiment is an example, and those skilled in the art will understand that various modifications can be made to the combination of each component or each treatment process, and that such modifications are also within the scope of the present disclosure. A modified example will be shown below.

A first modification will be explained. The sensor model unit 50 of the vehicle simulation system 110 applies coefficients according to the true value of the distance to an object (for example, a pedestrian) to the sensor model by curve fitting from a plurality of coefficients related to a plurality of dummy variables. .

FIG. 6 shows an example of curve fitting. Here, it is assumed that the sensor model includes at least 58 categories (explanatory variables). For example, the distance between a vehicle and a pedestrian may be categorized in 10 cm increments. The category coefficient graph 60 is a line graph showing the coefficients (category coefficients) of each of 58 categories of one explanatory variable (ie, error factor) in the sensor model. The error derivation unit 56 of the sensor model unit 50 derives an approximate curve 62 (also referred to as a quadratic polynomial approximate curve) by performing curve fitting on a plurality of category coefficients.

The error derivation unit 56 acquires a category coefficient corresponding to the true value of the distance from the vehicle to the pedestrian from the approximate curve 62. For example, if the distance is in 10 cm increments and the true value of the distance is 3.2 meters, the error derivation unit 56 uses the value of the approximate curve 62 (for example, "0") for the category number "32" as the category coefficient. You may obtain it. The error derivation unit 56 applies the category coefficient obtained from the approximate curve 62 as a weight of a dummy variable (in the example of FIG. 2, the value of the dummy variable is "1"), and derives the value of the objective variable (for example, detection distance error). You may. Alternatively, the true distance value may be converted into a coordinate on the horizontal axis as a continuous quantity, and the value calculated by substituting it into the approximate curve equation may be applied as the coefficient corresponding to the true distance value. According to these modifications, appropriate weighting of input explanatory variables can be achieved by curve fitting.

A second modification will be explained. The user interface 12 of the vehicle simulation system 110 may function as a reception unit that receives designation of data related to random numbers from the user. The simulated value deriving unit 58 of the sensor model unit 50 may add white noise to the estimated value using the sensor model based on data related to random numbers specified by the user.

For example, the user may input a seed for generating a pseudorandom number sequence to the user interface 12 as data related to random numbers. The simulated value deriving unit 58 generates a pseudorandom number sequence based on a seed input by the user, generates white noise data based on the pseudorandom number sequence, and estimates the distance between the vehicle and the pedestrian. You can also add white noise to the value. Since the data of the real environment includes a white noise component, according to this modification, it is possible to take the white noise into account and obtain an estimation result that is more in line with reality.

A third modification will be explained. In the embodiments described above, the distance between the vehicle and the pedestrian was estimated using a sensor model, but the technology described in the embodiments can be applied to the case where various conditions related to the surroundings of a vehicle are estimated using a model. In the third modification, an example will be shown in which the position of a parking frame (also referred to as a parking section) is estimated by a sensor model.

FIG. 7 shows the state around the vehicle in a modified example. In this modification, the parking frame 76 is defined by the XY coordinates of its four corners (FL, FR, BL, BR) and the direction of the parking frame 76 (the angle of the parking frame 76 with respect to the direction of movement of the vehicle, θ in FIG. 7). Expressed as In this modification, the coordinates of each of the four corners of the parking frame 76 detected by the image analysis unit are the distances of the four corners (FL, FR, BL, BR) from the camera 72 (for example, the distance of the front right point FR). Range _R ), the angle of the camera 72 with respect to the optical axis 74 (for example, β _R in the case of the front right point FR), and the direction of the parking frame 76 (θ). The sensor model generation device 20 of this modification categorizes each of the distance from the camera 72, the angle of the camera 72 with respect to the optical axis 74, and the direction of the parking frame 76.

Specifically, the image analysis unit 28 of the sensor model generation device 20 detects the position (coordinates, etc.) of the parking frame as the state of the surroundings of the vehicle from image data showing the surroundings of the vehicle. The true value detection unit 30 detects the true value of the position (coordinates, etc.) of the parking frame based on the detection result of the LIDAR device. The model generation unit 38 uses the coordinates of the parking frame 76 detected by the true value detection unit 30 (referred to as FR coordinates) and the coordinates of the parking frame 76 detected by the image analysis unit 28 (referred to as FR coordinates). The difference between is identified as the objective variable.

Further, the model generation unit 38 treats the distance from the camera 72, the angle of the camera 72 with the optical axis 74, and the angle of the direction θ of the parking frame 76 with the camera optical axis 74 as explanatory variables. Similar to the examples, these data are collected through tests using actual vehicles, and a plurality of samples are created. The model generation unit 38 executes multiple regression analysis based on a plurality of samples to obtain coefficients for each category (dummy variable). Note that, in reality, sensor models corresponding to the coordinates of each of the four corners of the parking frame 76 may be generated.

The simulation control unit 10 of the vehicle simulation system 110 receives the coordinates of the four corners (FL, FR, BL, BR) and the direction of the parking frame from the user as simulation parameters (the coordinate system may be the vehicle coordinate system, or the sensor coordinate system). system), and input those parameters to the sensor model section 50. The sensor model unit 50 estimates the detection result by the image analysis unit 28 by inputting simulation parameters into the sensor model, that is, estimates the coordinates of the parking frame 76. Specifically, the sensor model unit 50 first converts the input coordinates of the four corners (FL, FR, BL, BR) and the direction of the parking frame into explanatory variables such as the distance from the camera 72 and the light of the camera 72. The angle with respect to the axis 74 and the direction of the parking frame 76 are converted. Next, the sensor model unit 50 determines a category based on the obtained value of the explanatory variable, and applies a coefficient corresponding to the category. The vehicle model unit 18 simulates automatic travel of the vehicle (for example, automatic parking into the parking frame 76) based on the coordinates of the parking frame 76 estimated by the sensor model unit 50.

The techniques described in the first embodiment and the modified example may be specified by the following items.
[Item 1]
a first detection unit that detects a state of the surroundings of the vehicle for controlling automatic driving of the vehicle from image data showing the surroundings of the vehicle;
a second detection unit that detects a state around the vehicle with higher accuracy than the estimation unit;
A model for estimating the detection result by the first detection unit in the vehicle simulation system, wherein the state detected by the second detection unit is used as an explanatory variable, and the state detected by the second detection unit and , a generation unit that generates a model whose objective variable is a difference from the state detected by the first detection unit;
A model generation device comprising:
According to this model generation device, when the detection results from the first detection unit are required when simulating a vehicle, it is no longer necessary to prepare images for each case to be simulated and to analyze the images for each case. As a result, simulation costs can be reduced. Furthermore, by using the actual conditions around the vehicle in generating the model, it is possible to generate a model that outputs results consistent with reality.
[Item 2]
A model for estimating a detection result by the detection unit for detecting a state of the surroundings of the vehicle for controlling automatic driving of the vehicle from image data showing the surroundings of the vehicle, the model comprising: a storage unit that stores a model in which the true value of the state of the vehicle is used as an explanatory variable, and the difference between the true value of the state of the surroundings of the vehicle and the state detected by the detection unit is used as the objective variable;
an estimation unit that estimates a detection result by the detection unit by inputting a true value of the state around the vehicle as a simulation parameter to the model;
a simulation unit that simulates automatic driving of the vehicle based on the estimation result by the estimation unit;
A vehicle simulation system equipped with
According to this vehicle simulation system, if the detection results from the first detection unit are required when simulating a vehicle, it is no longer necessary to prepare images for each case to be simulated and to analyze the images for each case. As a result, simulation costs can be reduced. Further, by using a model generated using the actual conditions around the vehicle, it is possible to simulate the vehicle behavior in accordance with reality.
[Item 3]
The model includes a plurality of dummy variables corresponding to multiple levels of values of an explanatory variable that is a true value of a state around the vehicle,
The estimating unit calculates the continuous amount of the coefficient for the continuous value of the explanatory variable, which is the true value of the state around the vehicle, from each coefficient related to the plurality of dummy variables by curve fitting, and applying coefficients corresponding to continuous values of the explanatory variable from among them to the model;
Vehicle simulation system according to item 2.
According to this vehicle simulation system, appropriate weighting of explanatory variables can be achieved by curve fitting.
[Item 4]
further comprising a reception unit that accepts data specifications regarding random numbers from a user;
The estimation unit adds white noise to the estimated value using the model based on the data regarding the random number.
Vehicle simulation system according to item 2 or 3.
According to this vehicle simulation system, it is possible to obtain estimation results that are more realistic by adding white noise, and it is also possible to provide reproducibility of the simulation by making it possible to specify data related to random numbers. .
[Item 5]
a first detection unit detects a state around the vehicle for controlling automatic driving of the vehicle from image data showing the surroundings of the vehicle;
a second detection unit detects a state around the vehicle with higher accuracy than the first detection unit;
A model for estimating the detection result by the first detection unit in the vehicle simulation system, wherein the state detected by the second detection unit is used as an explanatory variable, and the state detected by the second detection unit and , generating a model whose objective variable is the difference from the state detected by the first detection unit;
Model generation method.
According to this model generation method, when the detection results from the first detection unit are required when simulating a vehicle, it is no longer necessary to prepare images for each case to be simulated and to analyze the images for each case. As a result, simulation costs can be reduced. Furthermore, by using the actual conditions around the vehicle in generating the model, it is possible to generate a model that outputs results consistent with reality.
[Item 6]
A model for estimating a detection result by the detection unit for detecting a state of the surroundings of the vehicle for controlling automatic driving of the vehicle from image data showing the surroundings of the vehicle, the model comprising: storing a model in which the true value of the state of the vehicle is used as an explanatory variable, and the difference between the true value of the state of the surroundings of the vehicle and the state detected by the detection unit is used as the objective variable;
Estimating the detection result by the detection unit by inputting the true value of the state around the vehicle as a simulation parameter to the model,
simulating automatic driving of the vehicle based on the estimation result by the estimation unit;
Vehicle simulation method.
According to this vehicle simulation method, when the detection results from the first detection unit are required when simulating a vehicle, it is no longer necessary to prepare images for each case to be simulated and to analyze the images for each case. As a result, simulation costs can be reduced. By using a model generated using the actual conditions around the vehicle, it is possible to simulate the vehicle behavior in accordance with reality.
[Item 7]
a first detection unit detects a state around the vehicle for controlling automatic driving of the vehicle from image data showing the surroundings of the vehicle;
a second detection unit detects a state around the vehicle with higher accuracy than the first detection unit;
A model for estimating the detection result by the first detection unit in the vehicle simulation system, wherein the state detected by the second detection unit is used as an explanatory variable, and the state detected by the second detection unit and , generating a model whose objective variable is the difference from the state detected by the first detection unit;
A computer program that causes a computer to do something.
According to this computer program, if the detection results from the first detection unit are required when simulating a vehicle, it becomes unnecessary to prepare images for each case to be simulated and to analyze the images for each case. , the cost of simulation can be reduced. Furthermore, by using the actual conditions around the vehicle in generating the model, it is possible to generate a model that outputs results consistent with reality.
[Item 8]
A model for estimating a detection result by the detection unit for detecting a state of the surroundings of the vehicle for controlling automatic driving of the vehicle from image data showing the surroundings of the vehicle, the model comprising: storing a model in which the true value of the state of the vehicle is used as an explanatory variable, and the difference between the true value of the state of the surroundings of the vehicle and the state detected by the detection unit is used as the objective variable;
Estimating the detection result by the detection unit by inputting the true value of the state around the vehicle as a simulation parameter to the model,
simulating automatic driving of the vehicle based on the estimation result by the estimation unit;
A computer program that causes a computer to do something.
According to this computer program, if the detection results from the first detection unit are required when simulating a vehicle, it becomes unnecessary to prepare images for each case to be simulated and to analyze the images for each case. , the cost of simulation can be reduced. By using a model generated using the actual conditions around the vehicle, it is possible to simulate the vehicle behavior in accordance with reality.

<Second example>
An outline of the second embodiment will be explained. In conventional vehicle simulation systems, only the CG recognition results or the true values of the conditions around the vehicle in the simulation (which can also be called ideal values; for example, the true positions of other vehicles near the own vehicle) are depicted in the simulation results. was. As a result, simulation results may not contribute sufficiently to the evaluation, problem extraction, and improvement of sensing algorithms and control algorithms.

Similarly to the vehicle simulation system of the first embodiment, the vehicle simulation system of the second embodiment estimates sensing results (including errors) using a sensor model. Further, the vehicle simulation system of the second embodiment displays the estimated value of the sensing result together with the true value of the sensing target. Furthermore, the vehicle simulation system of the second embodiment further outputs information regarding the error between the estimated value of the sensing result and the true value of the sensing target. According to the vehicle simulation system of the second embodiment, it is possible to provide more useful simulation results for evaluation, problem extraction, improvement, etc. of sensing algorithms and control algorithms.

Hereinafter, components of the second embodiment that are the same as or correspond to components of the first embodiment will be described with the same reference numerals as those of the first embodiment. In the second embodiment, the contents already explained in the first embodiment will not be explained again, and mainly the points different from the first embodiment will be explained.

Similarly to the sensor model of the first embodiment, the sensor model of the second embodiment uses the true values of multiple items related to the state of the surroundings of the vehicle as explanatory variables, and uses the detection results of the states of the surroundings of the vehicle by the detection unit mounted on the vehicle. This is a mathematical model (for example, a regression equation) in which the objective variable is the difference between the true value and the above true value. The detection unit is a sensor device mounted on the vehicle, and detects the conditions around the vehicle (for example, the positions of pedestrians and obstacles). The detection unit may include, for example, at least one of a camera, a radar device, a lidar device, and a sonar.

The sensor model of the second embodiment includes a sensor model for estimating a distance error (hereinafter also referred to as a "distance error model") and a sensor model for estimating a direction error (hereinafter also referred to as a "direction error model"). )including. The objective variable in the distance error model is the difference between the detection result and the true value regarding the distance from the vehicle to the object to be detected (for example, a pedestrian or an obstacle) (hereinafter also referred to as "distance error"). The objective variable in the azimuth error model is the difference between the detection result and the true value regarding the azimuth of the object to be detected with respect to the vehicle (hereinafter also referred to as "azimuth error"). The unit of distance error may be "meter". Further, the unit of the orientation error may be "degrees".

FIG. 8 shows an example of sample data for constructing a distance error model. The sample data is true values collected in an actual test environment using an actual vehicle. The value measured in the above test environment is set in the objective variable item "distance error" in the sample data. Items of explanatory variables of the distance error model include distance, direction, illuminance, and attitude. The distance is the distance from the vehicle to the detection target, specifically, the distance from the detection unit mounted on the vehicle to the detection target. The azimuth is the direction (also referred to as the direction) of the object to be detected with respect to the vehicle, and specifically, the angle between the optical axis of the detection unit mounted on the vehicle and the object to be detected (for example, the angle between -90° and 90°). value within the range). The illuminance is the illuminance around the vehicle. The posture is the posture of the object to be detected. FIG. 8 shows an example of a posture when the object to be detected is a pedestrian.

Similar to the sample data (FIG. 3) shown in the first example, each of the multiple explanatory variables includes multiple categories. For each sample, the value of the applicable category is set to "1" and the value of the non-applicable category is set to "0" based on the true value in an actual test environment using an actual vehicle. Note that when categorizing numerical explanatory variables (distance, direction, illuminance in FIG. 8), the increments do not have to be uniform. For example, in the case of illuminance, illuminance from 0 to 1000 lux may be set in increments of "10", and illuminance greater than 1000 lux may be set in increments of "100".

Regarding the sample data for constructing the orientation error model, the objective variable is the orientation error (actually measured value), but the explanatory variable items are the same as those for the distance error model. That is, the explanatory variables for constructing the orientation error model also include distance, direction, illuminance, and attitude, and each explanatory variable is divided into a plurality of categories. Here, if we generalize the number of items of explanatory variables as m, the number of categories of each item of explanatory variables as _n1 ,... _nm , and the number of samples as N, then the distance error model and the orientation error model are It can be expressed by Equation 1 described in the example.

The configuration of the sensor model generation device 20 of the second embodiment is similar to the configuration of the sensor model generation device 20 of the first embodiment (FIG. 3). The model generation unit 38 of the sensor model generation device 20 performs multiple regression analysis on the sample data of the distance error model, and calculates category coefficients (weights) for each category of each explanatory variable so that the sum of squares of residuals is minimized. Derive. The model generation unit 38 generates a regression equation (formula 1 above) in which the derived category coefficients are set as a distance error model, and stores data of the distance error model in the model storage unit 40. The method of generating the orientation error model is also similar.

In addition, the sensor model of the second embodiment is a sensor model (hereinafter also referred to as a "detection rate model") for estimating the detection rate (i.e., the rate at which objects that should be detected can be detected) by the detection unit installed in the vehicle. ). The detection rate model is a mathematical model in which the explanatory variables are the true values of multiple items related to the state around the vehicle, and the objective variable is whether or not the detection unit detects the state around the vehicle.

FIG. 9 shows an example of sample data for constructing a detection rate model. The explanatory variables for constructing the detection rate model are the same as those for the distance error model. That is, the explanatory variables for constructing the detection rate model also include distance, direction, illuminance, and posture, and each explanatory variable is divided into a plurality of categories. The objective variable for constructing the detection rate model is set to "1" when the detection is OK, and "0" when the detection is NG.

The sensor model generation device 20 of the second embodiment further includes a detection presence/absence determination section in order to generate a detection rate model. Instead of deriving the error by the error deriving part 32, the detection presence/absence determination part determines whether or not the object to be detected has been detected by the image analysis part 28 (that is, the sensing algorithm). When the detection presence/absence determination unit determines that detection has occurred, it sets "1" indicating detection OK in the sample data, and when determining that detection has not occurred, it sets "0" indicating detection NG in the sample data.

The model generation unit 38 of the sensor model generation device 20 generates a detection rate model using the same method as the distance error model and the direction error model. Specifically, the model generation unit 38 performs multiple regression analysis on the sample data of the detection rate model, and calculates the category coefficients (weights) for each category of each explanatory variable so that the sum of squares of residuals is minimized. Derive. The model generation unit 38 generates a regression equation (formula 1 above) in which the derived category coefficients are set as a detection rate model, and stores data of the detection rate model in the model storage unit 40. The detection rate model of the embodiment outputs a value closer to 0 as the detection possibility is lower, and outputs a value closer to 1 as the detection possibility increases.

As a modification, a large amount of sample data may be collected for each posture of the detection target. The model generation unit 38 may classify the sample data for each posture, and derive a detection rate (for example, an average value) for each combination of distance d, direction θ, and illuminance l based on the classified sample data. . The model generation unit 38 may generate a table (herein referred to as a "detection rate table") in which detection rates are arranged for each posture and for each combination of distance d, orientation θ, and illuminance l. FIG. 10 shows an example of a detection rate table for a certain posture.

The model generation unit 38 may use the detection rate table for each posture to generate a polynomial approximation curve with distance, direction, and illuminance as coefficients for each posture of the detection target. This polynomial approximate curve can be expressed by Equation 2.

The model generation unit 38 may derive a ₁ , a ₂ , b ₁ , b ₂ , c ₁ , c ₂ , and d ₀ of Equation 2 by polynomial approximation. The model generation unit 38 may store a plurality of polynomial approximate curves (Equation 2) corresponding to a plurality of postures of the detection target in the model storage unit 40 as a plurality of detection rate models. In this modification of the vehicle simulation, the detection rate is estimated using the detection rate model corresponding to the posture of the detection target among the plurality of detection rate models.

The configuration of the vehicle simulation system 110 of the second embodiment is similar to the configuration of the vehicle simulation system 110 of the first embodiment (FIG. 4). Further, the configuration of the sensor model section 50 of the second embodiment is also similar to the configuration of the sensor model section 50 of the first embodiment (FIG. 5). The sensor model unit 50 of the second embodiment has a sensor model unit 50 for detecting a state around the vehicle when the true value of the state around the vehicle is given as a simulation parameter from the simulation control unit 10. It functions as an estimator that estimates the detection result of the state of .

Specifically, the error deriving unit 56 of the sensor model unit 50 inputs the true value of the state around the vehicle as a simulation parameter to the distance error model, and obtains the distance error as an output from the distance error model. Further, the error deriving unit 56 inputs the true value of the state around the vehicle as a simulation parameter to the azimuth error model, and obtains the azimuth error as an output from the azimuth error model. The simulation parameters include the true values of the distance and direction from the detection unit to the object to be detected, the illuminance around the vehicle, and the attitude of the object to be detected.

Similar to the first embodiment, the simulated value derivation unit 58 of the sensor model unit 50 adds the error acquired by the error derivation unit 56 to the true value of the state around the vehicle in the simulation, thereby improving the detection by the detection unit. Derive estimates of the results. For example, the simulated value derivation unit 58 adds the distance error acquired by the error derivation unit 56 to the true distance value from the detection unit to the detection target in the simulation, thereby generating an estimated value of the distance detected by the detection unit. Derive. In addition, the simulated value deriving unit 58 adds the azimuth error acquired by the error deriving unit 56 to the true azimuth value of the detection target relative to the detection unit in the simulation, thereby calculating the estimated value of the azimuth detected by the detection unit. Derive.

Furthermore, the simulated value derivation unit 58 of the sensor model unit 50 inputs the true value of the state around the vehicle as a simulation parameter to the detection rate model, and derives an estimated value of the detection rate by the detection unit. For example, the simulated value deriving unit 58 inputs the illuminance around the vehicle and the posture of the detection target as simulation parameters to the detection rate model, and further calculates a combination of multiple patterns of distance and direction from the detection unit to the detection target. By inputting it to the detection rate model, an estimated detection rate value for each combination pattern of distance and direction is obtained as an output from the detection rate model. That is, the simulated value deriving unit 58 obtains a plurality of detection rate estimation values corresponding to a plurality of combinations of distance and direction patterns.

The simulated value deriving unit 58 outputs (1) information regarding distance estimation, (2) information regarding orientation estimation, and (3) information regarding detection rate estimation to the simulation control unit 10 as simulation results. (1) Information regarding distance estimation includes the estimated value of the distance detected by the detection unit, the true value of the distance in the simulation, the value of the distance error, and the breakdown of the distance error. (2) Information regarding orientation estimation includes an estimated value of the orientation detected by the detection unit, a true value of the orientation in the simulation, a value of the orientation error, and a breakdown of the orientation error. (3) Information regarding detection rate estimation includes a plurality of detection rate estimates corresponding to combinations of multiple patterns of distance and direction.

The breakdown of the distance error is the category coefficient of the category used in deriving the distance error among the multiple category coefficients for multiple categories in the distance error model (in other words, the category coefficient of the category to which the true value of the vehicle surrounding state corresponds). (hereinafter referred to as "effective category coefficients"). The breakdown of the distance error in the embodiment includes a plurality of sets of item names of explanatory variables in the distance error model (for example, "distance", "azimuth", etc.) and effective category coefficients. Similarly, the breakdown of the heading error is the category coefficient of the category used in deriving the heading error (in other words, the true value of the vehicle surrounding state is Contains category coefficients (effective category coefficients).The breakdown of the orientation error in the embodiment includes a plurality of pairs of item names of explanatory variables in the orientation error model and effective category coefficients.

The simulation control unit 10 of the second embodiment includes a function of generating an image showing the result of vehicle simulation. FIG. 11 is a block diagram showing functional blocks of the simulation control section 10 of the second embodiment. The simulation control section 10 includes a parameter acquisition section 120, a parameter input section 122, a simulated value acquisition section 124, a result image generation section 126, and a display control section 128.

A computer program including a plurality of modules in which the functions of the plurality of functional blocks shown in FIG. 11 are implemented may be stored in the storage of the vehicle simulation system 110. The CPU of vehicle simulation system 110 (or the CPU of a device within the system) may perform the functions of each functional block shown in FIG. 11 by reading this computer program into the main memory and executing it. Moreover, the plurality of functions of the plurality of functional blocks shown in FIG. 11 may be distributed to a plurality of devices, or may be realized by the plurality of devices working together as a system. Furthermore, multiple functions of multiple functional blocks shown in FIG. 11 may be integrated into a single device.

The parameter acquisition unit 120 acquires simulation parameters input by a user (developer, tester, etc.) to the user interface 12 (user terminal, etc.). The simulation parameters may be stored in advance in a predetermined storage section, and the parameter acquisition section 120 may read the simulation parameters from the storage section. As described above, the simulation parameters include the distance and direction from the detection unit to the object to be detected, the illuminance around the vehicle, and the posture of the object to be detected.

The parameter input unit 122 outputs the simulation parameters acquired by the parameter acquisition unit 120 to the sensor model unit 50, thereby causing the sensor model unit 50 to execute a simulation process. As described above, the sensor model unit 50 inputs the simulation parameters input from the simulation control unit 10 into the distance error model, the direction error model, and the detection rate model, and simulates the simulation results including the output values of each model. Output to the control unit 10.

The simulated value acquisition unit 124 acquires the results of the vehicle simulation performed by the sensor model unit 50. Specifically, the simulated value acquisition unit 124 acquires the above-mentioned (1) information related to distance estimation, (2) information related to azimuth estimation, and (3) information related to detection rate estimation, which are output from the sensor model unit 50.

Based on the simulation results acquired by the simulated value acquisition unit 124, the result image generation unit 126 generates an image (hereinafter also referred to as a “simulation result image”) representing the simulation results. The result image generation unit 126 generates a plurality of elements, which will be described later, included in the simulation result image. For example, the result image generation unit 126 generates, as the simulation result image, an image that shows the true value of the state around the vehicle in the simulation and the estimated value of the detection result by the vehicle detection unit in different ways.

The display control unit 128 outputs data of the simulation result image generated by the result image generation unit 126 to a display device (user interface 12 in the embodiment), and causes the display device to display the simulation result image.

The simulation result image will be explained in detail. FIG. 12 shows an example of a simulation result image. The simulation result image 140 includes a vehicle image 141, a detected position image 142, a true position image 144, a distance error indicator 146, a direction error indicator 148, and a detected range image 150 as a plurality of elements. The vehicle image 141 is an image showing a vehicle equipped with a detection unit to be simulated.

The detection position image 142 is an image showing a position at which the detection unit detects the detection target (a position estimated by simulation, and referred to as a "detection position"). The result image generation unit 126 determines a detected position based on the estimated distance value and the estimated direction value included in the simulation result, and arranges the detected position image 142 at the detected position. The true position image 144 is an image showing the true position (referred to as "true position") of the detection target in the simulation. The result image generation unit 126 determines the true position based on the true distance value and the true azimuth value as simulation parameters, and arranges the true position image 144 at the true position.

Furthermore, the detected position image 142 includes a line connecting the detected position and the position of the detection unit of the vehicle image 141. Further, the true position image 144 includes a line connecting the true position and the position of the detection unit of the vehicle image 141. The result image generation unit 126 sets the detected position image 142 and the true position image 144 in different forms, in other words, makes the detected position image 142 and the true position image 144 have different appearances. For example, the detected position image 142 and the true position image 144 may have different shapes, lines, or fills. Further, the detected position image 142 and the true position image 144 may have different colors, different sizes, dotted lines and solid lines, and presence/absence of shading. Thereby, in the simulation results, the detected position and the true position can be intuitively distinguished, and the difference can be easily understood.

The distance error indicator 146 is an image that indicates the magnitude of the distance error between the detected position and the true position. The distance error indicator 146 emphasizes the difference between the length of the line connecting the detection position and the position of the detection unit in the vehicle image 141 and the length of the line connecting the true position and the position of the detection unit in the vehicle image 141. (indicated by warning colors, bold lines, etc.). The result image generation unit 126 sets the distance error indicator 146 according to the distance error of the simulation result. This makes it easy to understand distance errors in simulation results.

The orientation error indicator 148 is an image that indicates the magnitude of the orientation error between the detected position and the true position. The orientation error indicator 148 emphasizes the difference between the orientation from the position of the detection unit in the vehicle image 141 to the detection position and the orientation from the position of the detection unit in the vehicle image 141 to the true position (alert color, thick line, etc.) This is shown in . The result image generation unit 126 sets the orientation error indicator 148 according to the orientation error of the simulation result. This makes it easy to understand orientation errors in simulation results.

The detection range image 150 is an image showing the detection range by the detection unit of the vehicle. The simulation control unit 10 of the embodiment further includes a detection range storage unit (not shown) that stores in advance data on the detection range of the detection unit, which is determined in advance according to the type of the detection unit and product characteristics. The result image generation unit 126 acquires data on the detection range of the vehicle's detection unit from the storage unit, and generates a detection range image 150 indicating the detection range (a fan-shaped range in FIG. 12). The result image generation unit 126 sets the color of each position in the detection range to a color according to the detection rate by the detection unit. This makes it easy to understand the detection rate, which may vary depending on the position of the object, in the vehicle simulation results.

Specifically, the result image generation unit 126 specifies the detection rate estimate for each position in the detection range according to a plurality of detection rate estimates corresponding to a plurality of combinations of distance and direction patterns as vehicle simulation results. do. The result image generation unit 126 sets a relatively low brightness color (dark color) at a position where the detection rate estimate is relatively low, while setting a relatively low brightness color (dark color) at a position where the detection rate estimate is relatively high. Set a color with high brightness (bright color). In other words, the result image generation unit 126 lowers the brightness of the color set at a position where the estimated detection rate is lower, and increases the brightness of the color set at the position where the estimated detection rate is higher. do. The detection range image 150 in FIG. 13 shows that the detection rate is high near the optical axis 152 of the detection unit, and the detection rate is low near the ends of the detection range.

Further, the simulation result image 140 is configured to further display a breakdown of the difference between the true position and the detected position in accordance with a predetermined operation by the user. FIG. 13 shows an example of a simulation result image. This figure shows that when an operation to select the orientation error indicator 148 is input in the simulation result image 140, an error breakdown 154, which is an image showing a breakdown of the orientation error, is displayed. For example, when the orientation error indicator 148 is moused over, the result image generation unit 126 generates the error details 154 in a display control script code (such as JavaScript (registered trademark)) that is provided to the user interface 12 together with the data of the simulation result image. You can also set a code to display a popup.

Note that the simulation result image 140 is configured to display an error breakdown 154 indicating a breakdown of the distance error when the distance error indicator 146 is selected.

The error breakdown 154 in the second embodiment indicates the weight of each explanatory variable of a plurality of items in the distance error model or the direction error model. As described above, the simulation result output from the sensor model unit 50 includes a plurality of pairs of item names of explanatory variables in the distance error model and effective category coefficients as details of the distance error. Furthermore, the simulation result includes a plurality of pairs of item names of explanatory variables in the orientation error model and effective category coefficients as the details of the orientation error. The result image generation unit 126 generates an error breakdown 154 indicating the breakdown of the distance error so as to indicate the ratio of effective category coefficients of each explanatory variable in the distance error model. The result image generation unit 126 generates an error breakdown 154 that shows the breakdown of the orientation error so as to indicate the ratio of effective category coefficients of each explanatory variable in the orientation error model. By displaying the error details 154, the factors that caused the difference between the detected position and the true position can be presented to the user in an easy-to-understand manner.

Although the error breakdown 154 in FIG. 13 shows the ratio of effective category coefficients for each explanatory variable in a pie chart, the error breakdown 154 may include different types of statistical charts. FIG. 14 shows a display example of the error breakdown 156 that shows the ratio of effective category coefficients of each explanatory variable using a stacked bar graph. The error breakdown 156 in FIG. 14 shows the breakdown of both the orientation error and the distance error. In this case, the same error breakdown 156 may be displayed regardless of whether the orientation error indicator 148 or the distance error indicator 146 is selected. Note that the error breakdown may indicate information other than the ratio of effective category coefficients of each explanatory variable of the model. For example, the error breakdown may indicate only the name of each explanatory variable of the model, or may indicate the value of the orientation error or the value of the distance error.

The present disclosure has been described above based on the second embodiment. The second embodiment is an example, and those skilled in the art will understand that various modifications can be made to the combination of each component or each treatment process, and that such modifications are also within the scope of the present disclosure.

In the second embodiment, the simulation result image is provided to the user interface 12 for display, but as a modification, the simulation result image may be stored in a predetermined storage device. For example, instead of or together with the display control unit 128, the simulation control unit 10 includes a result image storage unit that stores in a predetermined storage unit a plurality of simulation result images 140 showing the results of vehicle simulations in a plurality of situations. (not shown) may be further provided. The display control unit 128 may cause the user interface 12 to display one or more simulation result images designated by the user among the plurality of simulation result images stored in the storage unit.

The techniques described in the second embodiment and the modified examples may be specified by the following items.
[Item 2-1]
The detection unit detects the state of the surroundings of the vehicle, and an estimation unit that estimates the detection result of the state of the surroundings of the vehicle by the detection unit when a true value of the state of the surroundings of the vehicle is given as a simulation parameter. ,
As an image showing the result of the simulation, an image is generated that includes both the true value of the surrounding state of the vehicle and the estimated value of the detection result by the detection unit, and shows the true value and the estimated value in different ways. a generation unit to
A vehicle simulation system equipped with
According to this vehicle simulation system, it is possible to provide simulation results that allow easy comparison of the true value of the state of the surroundings of the vehicle and the detection result (estimated value) by the detection unit. Thereby, for example, it is possible to more effectively support evaluation and improvement of the detection algorithm used by the detection unit.
[Item 2-2]
The image showing the simulation result further includes an image showing a difference between the true value and the estimated value.
Vehicle simulation system described in item 2-1.
According to this vehicle simulation system, it is possible to provide simulation results that allow the user to intuitively understand the difference between the true value of the surrounding state of the vehicle and the detection result (estimated value) by the detection unit.
[Item 2-3]
The image showing the simulation result is configured to further display a breakdown of the difference between the true value and the estimated value in accordance with a predetermined operation.
Vehicle simulation system according to item 2-1 or 2-2.
According to this vehicle simulation system, it is possible to provide simulation results that allow detailed understanding of the difference between the true value of the state of the surroundings of the vehicle and the detection result (estimated value) by the detection unit.
[Item 2-4]
The estimation unit includes a sensor model in which true values of a plurality of items related to the state of the surroundings of the vehicle are used as explanatory variables, and a difference between the detection result of the state of the surroundings of the vehicle by the detection unit and the true value is used as an objective variable; Estimating the detection result by the detection unit by inputting true values of a plurality of items related to the surrounding state of the vehicle as simulation parameters,
The breakdown of the difference between the true value and the estimated value indicates the weight of explanatory variables of multiple items;
Vehicle simulation system described in item 2-3.
According to this vehicle simulation system, it is possible to provide simulation results that allow understanding of the main factors that caused the difference between the true value of the surrounding state of the vehicle and the detection result (estimated value) by the detection unit.
[Item 2-5]
The generation unit further generates an image indicating a detection range by the detection unit as an image indicating the result of the simulation, and sets a color at each position in the detection range to a color according to a detection rate by the detection unit. do,
The vehicle simulation system according to any one of items 1 to 4.
According to this vehicle simulation system, it is possible to provide simulation results that allow the user to intuitively understand the detection range and detection rate of the detection unit.
[Item 2-6]
With respect to a detection unit that detects a state of the surroundings of the vehicle, when a true value of the state of the surroundings of the vehicle is given as a simulation parameter, a detection result of the state of the surroundings of the vehicle by the detection unit is estimated;
As an image showing the result of the simulation, an image is generated that includes both the true value of the surrounding state of the vehicle and the estimated value of the detection result by the detection unit, and shows the true value and the estimated value in different ways. do,
A vehicle simulation method that is performed by a computer.
According to this vehicle simulation method, it is possible to provide a simulation result that allows easy comparison of the true value of the state of the surroundings of the vehicle and the detection result (estimated value) by the detection unit. Thereby, for example, it is possible to more effectively support evaluation and improvement of the detection algorithm used by the detection unit.
[Item 2-7]
With respect to a detection unit that detects a state of the surroundings of the vehicle, when a true value of the state of the surroundings of the vehicle is given as a simulation parameter, a detection result of the state of the surroundings of the vehicle by the detection unit is estimated;
As an image showing the result of the simulation, an image is generated that includes both the true value of the surrounding state of the vehicle and the estimated value of the detection result by the detection unit, and shows the true value and the estimated value in different ways. do,
A computer program that causes a computer to do something.
According to this computer program, it is possible to provide a simulation result that allows easy comparison of the true value of the state around the vehicle and the detection result (estimated value) by the detection unit. Thereby, for example, it is possible to more effectively support evaluation and improvement of the detection algorithm used by the detection unit.

<Third Example>
Regarding this embodiment, the following explanation will focus on the points that are different from the first embodiment and the second embodiment, and the explanation of the common points will be omitted. Among the components of this embodiment, components that are the same as or correspond to those of the first embodiment and the second embodiment will be described with the same reference numerals.

In the third embodiment, a sensing detection rate model (hereinafter also referred to as "detection rate model") is generated by a statistical processing method of quantitative analysis using detection results based on large-scale real environment data as a sensor model. . A vehicle simulation system using this detection rate model does not require image processing compared to a system using CG, and thus has a lower calculation cost. Another advantage is that it is easy to obtain simulation results that match reality.

The detection rate model of the third embodiment is a mathematical model that estimates the detection rate of a detection unit that detects the state of the surroundings of a vehicle from image data showing the surroundings of the vehicle. In addition, in the detection rate model, the true values of multiple items related to the conditions around the vehicle are used as explanatory variables, and the probability that the above-mentioned conditions around the vehicle are detected by the detection unit installed in the vehicle (i.e., "detection rate") is used as the objective variable. shall be. The above detection unit is a sensor device mounted on a vehicle, and may include, for example, at least one of a camera, a radar device, a lidar device, and a sonar.

FIG. 15 corresponds to FIG. 9 and shows an example of sample data for constructing a detection rate model. The explanatory variables for constructing the detection rate model include multiple factors that influence the detection rate: (1) distance from the vehicle (substantially the same as the detection unit) to the detection target; (2) detection from the vehicle. It includes the direction to the object, (3) illuminance around the vehicle, and (4) the attitude of the object to be detected. Each factor is divided into multiple categories. For each factor, the value of the applicable category is set to "1", and the value of the non-applicable category is set to "0". In addition, the objective variable for constructing the detection rate model is set to "1" when it is detected by the detection unit (in the case of detection OK), and "1" is set when it is not detected by the detection unit (in the case of detection NG). "0" is set.

A method for determining the detection rate in the third embodiment will be explained. Let the number of factors be K (K=4 in FIG. 15), and the number of categories for each factor be l ₁ , l ₂ , . . . , l _k . A group of samples detected by the detection unit among the plurality of samples is defined as group 1, and the number thereof is defined as _n1 . Further, among the plurality of samples, a group of samples that are not detected by the detection unit is defined as group 2, and the number thereof is defined as _n2 . The total number of samples is N=n ₁ +n ₂ .

The value that sample #j of group #r takes for category #α of factor #i is represented by a dummy variable x _{i (α)} ^{r (j)} . As mentioned above, if sample #j of group #r reacts to category #α of factor #i (yes), x _{i (α)} ^r(j) = 1, and does not react (no). In this case, x _i(α) ^r(j) =0.
i=1, 2,..., K; α=l ₁ , l ₂ ,..., l _k
r=1,2,...,M (group of samples); j=1,2,...,n _r
In the third embodiment, the samples are divided into two groups: detection OK and detection NG, and M=2.

ａ_ｉ（α）は要因（説明変数）＃ｉのカテゴリ＃αの係数（ウエイト）とも呼ぶ。 Here, the value y ^{r (j)} of sample #j of group #r can be generalized as shown in Equation 3.

a _{i (α)} is also called the coefficient (weight) of category #α of factor (explanatory variable) #i.

When constructing the detection rate model, an appropriate weight a _{i (α)} for each category is determined so that the sample value y ^{r (j)} that well separates each group classified by external criteria is calculated. Specifically, based on the ratio η 2 between the total variance σ ² of the value y ^{r (j)} of each sample and the between-group variance (in other words, the inter-class variance) σ _B ² , y that maximizes ^{η 2 is} used as a reference. The value of ^r(j) is determined, that is, the weight a _i(α) of each category is calculated.

That is, in the third embodiment, when data of a plurality of samples (including data regarding the state (true value) around the vehicle detected by the true value generation unit 200 described later) is input to the detection rate model, the detection rate is For multiple values output by the model, a detection rate model is generated by determining the weight of each explanatory variable so that the ratio of the total variance of the multiple values to the variance between group 1 and group 2 is maximized. do.
The above η ² (hereinafter also referred to as “correlation ratio”), which is a measure of the quality of separation between groups, is expressed by Equation 4.

式５中の^ーｘ_ｉ（α） ^ｒは、式６を意味する。

ｒは、１（検知ＯＫ）、または２（検知ＮＧ）である。
全体の平均は、式７で表される。

また、式７中の^ーｘ_ｉ（α）は、式８を意味する。

Here, the average within each group is expressed by Equation 5.

^−x _{i (α)} ^r in Formula 5 means Formula 6.

r is 1 (detection OK) or 2 (detection NG).
The overall average is expressed by Equation 7.

Moreover, ^−x _{i (α)} in Equation 7 means Equation 8.

グループ間分散σ_Ｂ ^２は、式１０で表される。

The total variance σ ² is expressed by Equation 9.

The intergroup variance σ _B ² is expressed by Equation 10.

From the above, the correlation ratio η ² becomes a function of L variables a _{g (β)} (g=1, 2, . . . , K, β=1, 2, . . . , l _g ). L is the number of all categories and is expressed by Equation 11.

Here, in order to obtain a _g(β) that maximizes the correlation ratio η ² as shown in Equation 12, if η ² is differentiated by a _g(β) and set to 0, Equation 13 is derived.

By solving Equation 13, the weight a _{i (α)} of each category that maximizes the correlation ratio η ² (i=1, 2,..., K, α=1, 2,..., l _i ) can be found.

To make it easier to understand how to obtain a i _(α) (i=1,2,...,K, α=1,2,...,l _i ), we will explain the calculation below using vectors and matrices. do.
First, let vector A be a vector consisting of category weights (Equation 14).

全サンプル平均値からなるＮ行Ｌ列の行列を^ーＸとする（式１６）。

全サンプルからなるＮ行Ｌ列の行列をＸとする（式１７）。

Next, a matrix of N rows and L columns consisting of intra-group average values is defined ^as _-XB (Equation 15).

^Let -X be a matrix of N rows and L columns consisting of all sample average values (Equation 16).

Let X be a matrix of N rows and L columns consisting of all samples (Equation 17).

即ち、式２１となる。

Furthermore, when two L-row L-column matrices S _B (Equation 18) and S (Equation 19) are created, Equation 13 is expressed by the following Equation 20:

That is, Equation 21 is obtained.

What is necessary is to solve Equation 21 and find A corresponding to the maximum η ² . We omit the detailed proof here. The sum of the dummy variables within each factor is 1 (Equation 21-2).

Therefore, since the rank of S does not become larger than (LK), the weight of the first category of each factor is set to 0 in the solution. Vector A, matrix X, ^−X _B , ^−X excluding the column corresponding to the first category of each factor are respectively denoted as A ^* , X ^* , ^−X _B ^* , ^{−X *} , and S _B ^*. When S ^* is expressed by Equation 22 and Equation 23, the equation corresponding to Equation 21 becomes Equation 24.

Next, η ² and A ^* are determined using Equation 24.
F is a lower triangular matrix that satisfies Equation 25, and from Equation 24, η ² is determined as the root (largest root) of the characteristic equation expressed as Equation 26.

On the other hand, A ^* is determined from the vector corresponding to the eigenvalue of Equation 26 (Equation 27). Here C is an eigenvector.
Here, by multiplying both sides of Equation 27 by F and substituting I=F'F' ^-1 , Equation 28 is obtained. By comparing Equation 28 with Equation 24, Equation 29 can be found. In other words, the category weight vector A ^* is obtained by equation 29.

式３０により求まる検知率ｙは、最小０、最大１の値にする、即ち、０より小さい値になった場合０とし、１より大きい値になった場合１とする。 Once the weight of each category is determined, the detection rate model is expressed by Equation 30.

The detection rate y determined by Equation 30 is set to a value of 0 at the minimum and 1 at the maximum, that is, set to 0 when the value is smaller than 0, and set to 1 when the value is larger than 1.

FIG. 16 is a block diagram showing functional blocks of the sensor model generation device 220 of the third embodiment. The sensor model generation device 220 includes a sensor data storage section 22, a reference data storage section 24, a tag data storage section 26, an image analysis section 28, a true value generation section 200, a detection result classification section 202, a true value quantization section 34, and a tag. It includes a quantization section 36 and a model generation section 38. A computer program in which the functions of at least some of these functional blocks are implemented may be installed in the storage of the sensor model generation device 220 via a predetermined recording medium, and may be installed in the storage of the sensor model generation device 220 via a network. It may be installed in the storage of the generation device 220. The CPU of the sensor model generation device 220 may perform the functions of each functional block by reading this computer program into the main memory and executing it.

The sensor data storage unit 22 and the reference data storage unit 24 store data collected during test runs using an actual vehicle. The sensor data storage unit 22 stores sensing results of various on-vehicle sensors regarding objects to be detected. In the embodiment, the sensor data storage unit 22 stores a plurality of image data captured by an on-vehicle camera, such as image data showing a pedestrian 3 meters in front of the left front of the vehicle, image data in the front direction of the vehicle, etc. Image data showing a pedestrian 6 meters ahead, etc. are stored.

The reference data storage unit 24 stores true value detection data (also referred to as "reference data") regarding the object to be detected. In the embodiment, the reference data storage unit 24 stores, as reference data, data collected by LIDAR (Light Detection and Ranging), which substantially indicates the position of the object (distance and direction from the vehicle to the object). Store data regarding true values.

The tag data storage unit 26 stores data set by a user (for example, a developer, a tester, etc.) as tag data. The tag data of the third example includes data indicating the posture of the object to be detected and data indicating the illuminance around the vehicle during the test run. Note that the tag data may include various data such as weather, type of road surface (for example, asphalt or soil), temperature, season, etc.

The image analysis unit 28 corresponds to a sensor device mounted on a vehicle, and detects the state around the vehicle to control the behavior of the vehicle. The image analysis section 28 serves as a first detection section and detects the state of the surroundings of the vehicle based on the image data of the surroundings of the vehicle stored in the sensor data storage section 22 . For example, the image analysis unit 28 detects the position of the object to be detected as the state around the vehicle, and specifically detects the distance and direction from the vehicle of people or objects existing around the vehicle.

The true value generating section 200 corresponds to the true value detecting section 30 of the first embodiment. The true value generation section 200, as a second detection section, detects the state around the vehicle with higher accuracy than the image analysis section 28 based on the reference data stored in the reference data storage section 24. Like the image analysis unit 28, the true value generation unit 200 also detects the position of the object to be detected as the state around the vehicle, and specifically, detects the distance and direction of people and objects around the vehicle from the vehicle. Detect.

The true value quantization unit 34 quantizes data regarding the state around the vehicle detected by the true value generation unit 200. For example, the true value quantization unit 34 classifies the detection results (for example, the distance and direction from the vehicle to the object) by the true value generation unit 200 into one of a plurality of predetermined explanatory variable categories. As shown in FIG. 15, the true value quantization unit 34 sets the value (dummy variable) of the category corresponding to the true value of the distance and direction from the vehicle to the target object to "1", and sets the value of the category that does not correspond to the true value of the distance and direction from the vehicle to "1", and Set the value (dummy variable) to "0".

The tag quantization unit 36 quantizes the tag data stored in the tag data storage unit 26. For example, the tag quantization unit 36 classifies the posture and illuminance of the detection target into one of a plurality of categories of predetermined explanatory variables. That is, the value of the applicable category (dummy variable) is set to "1", and the value of the non-applicable category (dummy variable) is set to "0".

The detection result classification unit 202 determines, for each sample, whether the state around the vehicle detected by the true value generation unit 200 is also detected by the image analysis unit 28. For example, if the difference between the distance and direction of the object detected by the true value generation section 200 and the distance and direction of the object detected by the image analysis section 28 is within a predetermined threshold, the detection result classification section 202 , it may be determined that the state around the vehicle detected by the true value generation unit 200 is detected by the image analysis unit 28. On the other hand, if the difference exceeds the threshold, the detection result classification section 202 may determine that the state around the vehicle detected by the true value generation section 200 is not detected by the image analysis section 28. An appropriate value for the above threshold value may be determined through an experiment using the vehicle simulation system 110 (sensor model unit 50) of the third embodiment.

The detection result classification unit 202 classifies each sample into group 1 (i.e., the condition detected by the image analysis unit 28 The images are classified into either Group 2 (things not detected by the image analysis unit 28). When a certain sample is classified into group 1, the detection OK/NG value of that sample data is set to "1", and when it is classified into group 2, the detection OK/NG value of that sample data is set to "0". Set.

The above processing of the image analysis unit 28, true value generation unit 200, true value quantization unit 34, tag quantization unit 36, and detection result classification unit 202 is performed on image data and reference data collected at the same timing, and their It is executed for each sample, which is a combination of tag data at a timing. The model generation unit 38 generates a detection rate model based on a plurality of sample data in the format shown in FIG.

The model generation unit 38 uses the state around the vehicle detected by the true value generation unit 200 as an explanatory variable, and calculates the probability that the state around the vehicle detected by the true value generation unit 200 is detected by the image analysis unit 28. Generate a detection rate model with variables. Specifically, the model generation unit 38 calculates the weights a _{i( α)} is calculated to generate a detection rate model (Equation 30) that maximizes the correlation ratio η ² . The model generation unit 38 stores the generated detection rate model in the model storage unit 40.

Next, a vehicle simulation system according to a third embodiment that uses the sensor models (distance error model, direction error model, and detection rate model) generated by the sensor model generation device 220 will be described. The configuration of the vehicle simulation system 110 of the third embodiment is similar to the configuration of the vehicle simulation system 110 of the first embodiment shown in FIG.

The sensor model unit 50 of the vehicle simulation system 110 of the third embodiment generates a sensor generated in advance by the sensor model generation device 220 when the true value of the state around the vehicle is given as a simulation parameter from the simulation control unit 10. The model is used to estimate the detection results of the detection unit that detects the conditions around the vehicle.

FIG. 17 is a block diagram showing functional blocks of the sensor model section 50 of the third embodiment. The sensor model section 50 includes a model storage section 40, a true value quantization section 54, a tag quantization section 52, an error derivation section 56, a detection result generation section 210, a detection rate calculation section 212, and an output section 214. A computer program in which the functions of at least some of these functional blocks are implemented is stored in the storage of the sensor model unit 50 (or an information processing device that implements the sensor model unit 50) via a predetermined recording medium. It may be installed or may be installed in the storage of the sensor model unit 50 via a network. The CPU of the sensor model unit 50 may perform the functions of each functional block by reading this computer program into the main memory and executing it.

The model storage unit 40 stores the distance error model and direction error model (also collectively referred to as "error model") described in the second embodiment, and the detection rate model described in the third embodiment.

The true value quantization unit 54 receives true value data regarding the state around the vehicle (specifically, the distance and direction of the object) as simulation parameters input from the simulation control unit 10. Similar to the process of the true value quantization unit 34, the true value quantization unit 54 quantizes the received true value data and identifies a corresponding category from among multiple categories of explanatory variables predetermined for each sensor model. do. Specifically, as input data to the error model and detection rate model, the true value quantization unit 54 sets the value of the corresponding category to "1" for the explanatory variables of each model, such as distance and direction, and sets the value of the applicable category to "1", and indicates that it is not applicable. Generate input data with the category value as "0".

The tag quantization unit 52 receives tag data related to the state around the vehicle (specifically, the posture and illuminance of the object) as simulation parameters input from the simulation control unit 10 . The tag quantization unit 52 quantizes the received tag data and identifies a corresponding category from among a plurality of categories of explanatory variables predetermined for each sensor model. Specifically, as input data to the error model and detection rate model, the tag quantization unit 52 sets the value of the applicable category to "1" for illuminance and posture, which are explanatory variables of each model, and sets the value of the non-applicable category to "1". Generate input data with the category value as "0".

The error derivation unit 56 converts the input data generated by the true value quantization unit 54 by quantizing the true value data and the input data generated by the tag quantization unit 52 by quantizing the tag data into an error model. input and obtain the distance and orientation errors output from the error model. Specifically, for the plurality of categories of the distance error model, the error derivation unit 56 sets the value of the dummy variable of the category that corresponds to the value of the true value data and tag data to 1, and sets the value of the dummy variable of the category that does not correspond to the value of the true value data and the tag data. By setting the value to 0, the distance error, which is the objective variable of the distance error model, is calculated using the weight of the category whose dummy variable value is 1. The method for calculating the orientation error using the orientation error model is also similar.

The detection result generation unit 210 corresponds to the simulated value derivation unit 58 of the first embodiment and the second embodiment, and is a first estimation unit that estimates the detection result based on the image by the detection unit (image analysis unit 28). functions as Specifically, the detection result generation unit 210 adds the distance error and direction error acquired by the error derivation unit 56 to true value data regarding the state around the vehicle (distance and direction of the object) as simulation parameters. By doing so, an estimated value of the detection result by the detection unit (image analysis unit 28) is generated.

The detection rate calculation unit 212 inputs the quantized data of the true value data of the state around the vehicle and the quantized data of the tag data as simulation parameters to the detection rate model. ) functions as a second estimator that estimates the detection rate based on the image. Specifically, the detection rate calculation unit 212 sets the value of the dummy variable of the category corresponding to the true value data and the tag data value to 1 for the plurality of categories of the plurality of explanatory variables of the detection rate model, and By setting the value of the dummy variable of the corresponding category to 0, the detection rate, which is the objective variable of the detection rate model, is calculated using the weight of the category whose dummy variable value is 1.

The output unit 214 performs simulation control on the estimated value of the detection result by the detection unit (image analysis unit 28), which is generated by the detection result generation unit 210, according to the estimated detection rate calculated by the detection rate calculation unit 212. It is determined whether or not to output to section 10. For example, in a simulation in which a pedestrian exists around a vehicle, if the detection rate calculation unit 212 estimates that the detection rate of the pedestrian is 30%, the output unit 214 outputs the detection result by the detection unit (image analysis unit 28). The estimated value (distance and direction of the pedestrian) may be transmitted to the simulation control unit 10 with a probability of 30%.

As described above, the estimated value of the detection result by the detection unit (image analysis unit 28) output from the sensor model unit 50 is input to the vehicle model unit 18 via the simulation control unit 10. In other words, the sensor model section 50 outputs the estimated value of the detection result by the detection section (image analysis section 28) to the vehicle model section 18 via the simulation control section 10. Therefore, the vehicle model unit 18 determines the behavior of the vehicle in the simulation according to the estimated value of the detection rate calculated by the detection rate calculation unit 212. In other words, the detection rate calculated by the detection rate calculation unit 212 is reflected in the simulation result by the vehicle model unit 18.

The operation of the above configuration will be explained.
First, an example of the operation of the sensor model generation device 220 will be described with reference to FIG. 16. Developers of in-vehicle equipment who are supposed to provide vehicle simulation systems to customers and partner companies actually drive the vehicle, capture images of pedestrians outside the vehicle with the in-vehicle camera, and store multiple samples of captured images in the sensor data storage unit 22. Make me remember. At the same time, the developer causes the LIDAR device to measure the position of the pedestrian, and stores position information (true values) of a plurality of samples in the reference data storage unit 24. Further, the developer causes the tag data storage unit 26 to store multiple samples of tag data including the illuminance and the posture of the pedestrian.

The image analysis unit 28 detects the distance and direction to the pedestrian for each sample based on the image stored in the sensor data storage unit 22. The true value generation unit 200 detects the true values of the distance and direction to the pedestrian based on the position information (true value) stored in the reference data storage unit 24 for each sample. The true value quantization unit 34 identifies the true value category of the distance and direction to the pedestrian for each sample. The tag quantization unit 36 identifies the category of tag data for each sample.

The detection result classification unit 202 classifies each sample into group 1 (those whose detection results by the true value generation unit 200 were also detected by the image analysis unit 28) and group 2 (those whose detection results by the true value generation unit 200 were also detected by the image analysis unit 28). classified as either (undetected). The detection result classification unit 202 sets "1" to group 1 and "0" to group 2 as the "detection OK/NG" value of each sample.

The model generation unit 38 uses the quantized data of the detection result by the true value generation unit 200 and the quantized data of the tag data as explanatory variables, and uses the value of “detection OK/NG” as an objective variable, and calculates the between-group variance versus total Generate a detection rate model so that the ratio of variance is maximized. The model generation unit 38 stores the detection rate model in the model storage unit 40. According to the sensor model generation device 220 of the third embodiment, an appropriate detection rate model that can estimate the detection rate in accordance with the actual environment and in which group 1 and group 2 are well separated is created. can be generated.

Note that the sensor model generation device 220 of the third embodiment further generates a distance error model and a direction error model and stores them in the model storage unit 40 using a method similar to that of the second embodiment.

Next, an example of the operation of the vehicle simulation system 110 will be described with reference to FIGS. 4 and 17. Here, similarly to the first embodiment, the behavior of the vehicle when a pedestrian exists near the vehicle will be simulated. The user inputs the position and posture of the pedestrian and the illuminance around the vehicle to the user interface 12 as simulation parameters. The environmental data generation unit 14 generates true value data indicating the distance and direction from the vehicle to the pedestrian based on the data received by the user interface 12, and also generates tag data indicating the posture and illuminance of the pedestrian. generate.

The simulation control unit 10 inputs the true value data and tag data generated by the environmental data generation unit 14 to the sensor model unit 50. The true value quantization unit 54 of the sensor model unit 50 specifies the category of true value data (for example, the distance and direction in FIG. 15) and generates quantized data of the true value data. The tag quantization unit 52 of the sensor model unit 50 specifies the category of tag data (for example, illuminance and posture in FIG. 15) and generates quantized data of the tag data.

The error deriving unit 56 of the sensor model unit 50 obtains a distance error by inputting the quantized data of the true value data and the quantized data of the tag data to a distance error model. Further, the error deriving unit 56 obtains the orientation error by inputting the quantized data of the true value data and the quantized data of the tag data to the orientation error model. The detection result generation unit 210 of the sensor model unit 50 adds a distance error and a direction error to the true value data indicating the distance and direction from the vehicle to the pedestrian, which is input from the simulation control unit 10. Generate an estimate of the detection result by.

The detection rate calculation unit 212 of the sensor model unit 50 calculates the detection rate by the vehicle detection unit by inputting the quantized data of the true value data and the quantized data of the tag data to the detection rate model. The output unit 214 of the sensor model unit 50 outputs the detection result by the vehicle detection unit generated by the detection result generation unit 210 with a probability based on the detection rate (a value of 0 or more and 1 or less) calculated by the detection rate calculation unit 212. The estimated value of the result is output to the simulation control section 10.

The simulation control unit 10 inputs the estimated value of the detection result by the vehicle detection unit output from the sensor model unit 50 to the vehicle model unit 18. The vehicle model section 18 simulates the behavior of the vehicle based on the estimated value of the detection result by the vehicle's detection section. The simulation control unit 10 displays the simulation results by the vehicle model unit 18 on the user interface 12 or stores them in a predetermined storage device.

According to the vehicle simulation system 110 of the third embodiment, there is no need to recognize the state around the vehicle through image processing, so calculation costs can be reduced. Furthermore, by using a detection rate model generated based on the actual conditions around the vehicle, it is possible to obtain realistic simulation results. Furthermore, model-based development of in-vehicle devices and the like can be made even more efficient.

The present disclosure has been described above based on the third embodiment. It will be understood by those skilled in the art that this example is merely an illustration, and that various modifications can be made to the combination of each component or each treatment process in the example, and that such modifications are also within the scope of the present disclosure. be.

A modification will be explained. Although not mentioned in the third embodiment, the sensor model section 50 in FIG. 4 may adopt the structure of the functional block diagram in FIG. 18. In FIG. 18, the detection rate calculation unit 212 calculates the detection rate as described in paragraph 0152, and outputs it to the error derivation unit 56. The error derivation unit 56 uses the estimated value of the detection rate calculated by the detection rate calculation unit 212 as a probability, and determines whether to perform the error calculation process described in paragraph 0150. When the error derivation unit 56 determines that the detection was not possible based on the probability, the error derivation unit 56 outputs a result of “undetectable” to the detection result generation unit 210. On the other hand, if the result is that the detection was possible, the error derivation unit 56 performs the processing described in paragraph 0150, calculates the distance error and the direction error, and sends the results to the detection result generation unit along with the result that the detection was possible. Output to 210. When the detection result generation unit 210 receives the result that “detection was successful”, it generates and outputs an estimated value of the detection result by the detection unit (image analysis unit 28) using the process described in paragraph 0151. On the other hand, if "unable to detect" is input, the detection result generation unit 210 outputs a detection result of "empty".

The detection rate model generation method described in the third embodiment is also applicable to the second embodiment. For example, if the detection rate model generated by the method described in the third embodiment is applied to the vehicle simulation system 110 of the second embodiment, and the simulation result image (FIG. 12 etc.) described in the second embodiment is generated. good.

The techniques described in the third embodiment and the modified example may be specified by the following items.
[Item 3-1]
a first detection unit that detects the state of the surroundings of the vehicle from image data showing the surroundings of the vehicle;
a second detection unit that detects a state around the vehicle with higher accuracy than the first detection unit;
A model for estimating a detection rate by the first detection unit in the vehicle simulation system, wherein the state detected by the second detection unit is used as an explanatory variable, and the state detected by the second detection unit is , a generation unit that generates a model whose objective variable is the probability of being detected by the first detection unit;
A model generation device comprising:
According to this model generation device, it is possible to generate a model for realizing a vehicle simulation that reflects the detection rate by the first detection unit. Furthermore, by using image data showing the surroundings of the vehicle in generating the model, it is possible to realize a model that outputs a detection rate that is consistent with reality. Furthermore, in vehicle simulation, it is no longer necessary to prepare images for each case, and it is no longer necessary to analyze images for each case, making it possible to reduce simulation costs.
[Item 3-2]
The generation unit generates the model based on a plurality of samples including data regarding the state detected by the second detection unit,
For the plurality of samples, a state detected by the second detection unit is detected by the first detection unit, and a state detected by the second detection unit is detected by the first detection unit. When classifying into the second group that is not
variance between the first group and the second group with respect to the plurality of values output by the model when data regarding the states detected by the second detection unit of the plurality of samples are input to the model; and generating the model so that the ratio of the total variance of the plurality of values is maximized;
The model generation device described in item 3-1.
According to this model generation device, it is possible to generate an appropriate model in which the first group and the second group are well separated.
[Item 3-3]
A model for estimating the detection rate of the detection unit for detecting the state of the surroundings of the vehicle from image data showing the surroundings of the vehicle, wherein the true value of the state of the surroundings of the vehicle is used as an explanatory variable; a storage unit that stores a model whose objective variable is a probability that a state around the vehicle is detected by the detection unit;
an estimation unit that estimates a detection rate by the detection unit by inputting a true value of the state around the vehicle as a simulation parameter into the model;
a simulation unit that simulates the behavior of the vehicle according to the detection rate estimated by the estimation unit;
A vehicle simulation system equipped with
According to this vehicle simulation system, there is no need to recognize the state around the vehicle through image processing, so calculation costs can be reduced. Furthermore, by using a model generated based on the actual conditions around the vehicle, it is possible to obtain realistic and useful simulation results. Furthermore, model-based development of in-vehicle devices and the like can be made even more efficient.
[Item 3-4]
The estimation unit further estimates the detection result by the detection unit, and determines whether to output the estimated value of the detection result by the detection unit to the simulation unit according to the estimated detection rate by the detection unit. do,
Vehicle simulation system described in item 3-3.
According to this vehicle simulation system, it is possible to obtain realistic and useful simulation results that reflect the detection rate of the detection unit.
[Item 3-5]
a first detection unit detects a state around the vehicle from image data showing the surroundings of the vehicle;
a second detection unit detects a state around the vehicle with higher accuracy than the first detection unit;
A model for estimating a detection rate by the first detection unit in the vehicle simulation system, wherein the state detected by the second detection unit is used as an explanatory variable, and the state detected by the second detection unit is , generating a model whose objective variable is the probability of being detected by the first detection unit;
Model generation method.
According to this model generation method, it is possible to generate a model for realizing a vehicle simulation that reflects the detection rate by the first detection unit. Furthermore, by using image data showing the surroundings of the vehicle in generating the model, it is possible to realize a model that outputs a detection rate that is consistent with reality. Furthermore, in vehicle simulation, it is no longer necessary to prepare images for each case, and it is no longer necessary to analyze images for each case, making it possible to reduce simulation costs.
[Item 3-6]
A model for estimating the detection rate of the detection unit for detecting the state of the surroundings of the vehicle from image data showing the surroundings of the vehicle, wherein the true value of the state of the surroundings of the vehicle is used as an explanatory variable; storing a model whose objective variable is a probability that a state around the vehicle is detected by the detection unit;
Estimating the detection rate by the detection unit by inputting the true value of the state around the vehicle as a simulation parameter to the model,
simulating the behavior of the vehicle based on the estimated detection rate;
Vehicle simulation method.
According to this vehicle simulation method, there is no need to recognize the state around the vehicle through image processing, so calculation costs can be reduced. Furthermore, by using a model generated based on the actual conditions around the vehicle, it is possible to obtain realistic and useful simulation results. Furthermore, model-based development of in-vehicle devices and the like can be made even more efficient.
[Item 3-7]
a first detection unit detects a state around the vehicle from image data showing the surroundings of the vehicle;
a second detection unit detects a state around the vehicle with higher accuracy than the first detection unit;
A model for estimating a detection rate by the first detection unit in the vehicle simulation system, wherein the state detected by the second detection unit is used as an explanatory variable, and the state detected by the second detection unit is , generating a model whose objective variable is the probability of being detected by the first detection unit;
A computer program that causes a computer to do something.
According to this computer program, it is possible to generate a model for realizing a vehicle simulation that reflects the detection rate by the first detection unit. Furthermore, by using image data showing the surroundings of the vehicle in generating the model, it is possible to realize a model that outputs a detection rate that is consistent with reality. Furthermore, in vehicle simulation, it is no longer necessary to prepare images for each case, and it is no longer necessary to analyze images for each case, making it possible to reduce simulation costs.
[Item 3-8]
A model for estimating the detection rate of the detection unit for detecting the state of the surroundings of the vehicle from image data showing the surroundings of the vehicle, wherein the true value of the state of the surroundings of the vehicle is used as an explanatory variable; storing a model whose objective variable is a probability that a state around the vehicle is detected by the detection unit;
Estimating the detection rate by the detection unit by inputting the true value of the state around the vehicle as a simulation parameter to the model,
simulating the behavior of the vehicle based on the estimated detection rate;
A computer program that causes a computer to do something.
According to this computer program, there is no need to recognize the state around the vehicle through image processing, so calculation costs can be reduced. Furthermore, by using a model generated based on the actual conditions around the vehicle, it is possible to obtain realistic and useful simulation results. Furthermore, model-based development of in-vehicle devices and the like can be made even more efficient.

Any combination of the embodiments and variations described above are also useful as embodiments of the present disclosure. A new embodiment resulting from a combination has the effects of each of the combined embodiments and modifications. Further, those skilled in the art will also understand that the functions to be fulfilled by the respective constituent elements described in the claims are realized by each constituent element shown in the embodiments and modified examples alone or by their cooperation.

18 vehicle model section, 28 image analysis section, 38 model generation section, 40 model storage section, 50 sensor model section, 110 vehicle simulation system, 200 true value generation section, 212 detection rate calculation section, 220 sensor model generation device.

Claims (8)

a first detection unit that detects the state of the surroundings of the vehicle from image data showing the surroundings of the vehicle;
a second detection unit that detects a state around the vehicle with higher accuracy than the first detection unit;
A model for estimating a detection rate by the first detection unit in the vehicle simulation system, wherein the state detected by the second detection unit is used as an explanatory variable, and the state detected by the second detection unit is , a generation unit that generates a model whose objective variable is the probability of being detected by the first detection unit;
A model generation device comprising: The generation unit generates the model based on a plurality of samples including data regarding the state detected by the second detection unit,
For the plurality of samples, a state detected by the second detection unit is detected by the first detection unit, and a state detected by the second detection unit is detected by the first detection unit. When classifying into the second group that is not
variance between the first group and the second group with respect to the plurality of values output by the model when data regarding the states detected by the second detection unit of the plurality of samples are input to the model; and generating the model so that the ratio of the total variance of the plurality of values is maximized;
The model generation device according to claim 1. A model for estimating the detection rate of the detection unit for detecting the state of the surroundings of the vehicle from image data showing the surroundings of the vehicle, wherein the true value of the state of the surroundings of the vehicle is used as an explanatory variable; a storage unit that stores a model whose objective variable is a probability that a state around the vehicle is detected by the detection unit;
an estimation unit that estimates a detection rate by the detection unit by inputting a true value of the state around the vehicle as a simulation parameter into the model;
a simulation unit that simulates the behavior of the vehicle according to the detection rate estimated by the estimation unit;
A vehicle simulation system equipped with The estimation unit further estimates the detection result by the detection unit, and determines whether to output the estimated value of the detection result by the detection unit to the simulation unit according to the estimated detection rate by the detection unit. do,
The vehicle simulation system according to claim 3. a first detection unit detects a state around the vehicle from image data showing the surroundings of the vehicle;
a second detection unit detects a state around the vehicle with higher accuracy than the first detection unit;
A model for estimating a detection rate by the first detection unit in the vehicle simulation system, wherein the state detected by the second detection unit is used as an explanatory variable, and the state detected by the second detection unit is , generating a model whose objective variable is the probability of being detected by the first detection unit;
Model generation method. A model for estimating the detection rate of the detection unit for detecting the state of the surroundings of the vehicle from image data showing the surroundings of the vehicle, wherein the true value of the state of the surroundings of the vehicle is used as an explanatory variable; storing a model whose objective variable is a probability that a state around the vehicle is detected by the detection unit;
Estimating the detection rate by the detection unit by inputting the true value of the state around the vehicle as a simulation parameter to the model,
simulating the behavior of the vehicle based on the estimated detection rate;
Vehicle simulation method. a first detection unit detects a state around the vehicle from image data showing the surroundings of the vehicle;
a second detection unit detects a state around the vehicle with higher accuracy than the first detection unit;
A model for estimating a detection rate by the first detection unit in the vehicle simulation system, wherein the state detected by the second detection unit is used as an explanatory variable, and the state detected by the second detection unit is , generating a model whose objective variable is the probability of being detected by the first detection unit;
A computer program that causes a computer to do something. A model for estimating the detection rate of the detection unit for detecting the state of the surroundings of the vehicle from image data showing the surroundings of the vehicle, wherein the true value of the state of the surroundings of the vehicle is used as an explanatory variable; storing a model whose objective variable is a probability that a state around the vehicle is detected by the detection unit;
Estimating the detection rate by the detection unit by inputting the true value of the state around the vehicle as a simulation parameter to the model,
simulating the behavior of the vehicle based on the estimated detection rate;
A computer program that causes a computer to do something.

Images