KR102340120B1 - Evaluation system for driving simulation and method thereof - Google Patents

Abstract

Various embodiments of the present disclosure relate to a driving simulation evaluation system and a method thereof, wherein the evaluation system performs a driving simulation, a simulator that generates driving simulation data, and an autonomous driving module based on the driving simulation data and a data generation module that generates learning data and evaluation data for performing the test, provides the learning data and the evaluation data to the autonomous driving module, and a verification module that verifies the autonomous driving module. Other embodiments are possible.

Description

Driving simulation evaluation system and method

Various embodiments of the present invention relate to a driving simulation evaluation system and a method thereof.

Vehicles have been developed in a direction in which electronic devices are coupled to increase user convenience as a means of providing rapid mobility. Along with the development of such vehicles, an intelligent transport system (ITS) technology linked to a road is being developed, and recently, a cooperative ITS technology in which a vehicle and road infrastructure are fused is being actively studied.

The background technology of the present invention is disclosed in Republic of Korea Patent Publication No. 10-2017-0117270 (published on October 23, 2017, V2X communication-based driving integrated simulation device).

Recently, as vehicles and artificial intelligence (AI) technologies are combined, autonomous driving modules (eg, AI platforms, AI autonomous driving modules) are being developed to support autonomous driving of vehicles, and SILS (software in the loop) that evaluates them is being developed. system) and HILS (hardware in the loop system) are also increasing interest. Therefore, a method for evaluating an autonomous driving module using SILS and HILS may be required.

Various embodiments of the present invention disclose an evaluation system and evaluation method for evaluating an autonomous driving module using SILS and HILS.

An evaluation system according to various embodiments of the present disclosure performs a driving simulation and generates a simulator for generating data of a driving simulation, and generates learning data and evaluation data for learning an autonomous driving module based on the data of the driving simulation, , a data generation module that provides the learning data and the evaluation data to the autonomous driving module, and a verification module that verifies the autonomous driving module.

A method of operating an evaluation system according to various embodiments of the present disclosure may include generating data of the driving simulation while a simulator of the evaluation system performs a driving simulation, and a data generating module of the evaluation system is configured to apply the driving simulation data to the data of the driving simulation. generating learning data and evaluation data for learning the autonomous driving module based on the steps; providing, by the data generation module, the learning data and the evaluation data to the autonomous driving module; and a verification module of the evaluation system It may include verifying the autonomous driving module.

Various embodiments of the present invention can more effectively evaluate the autonomous driving module by providing an environment for evaluating the autonomous driving module using SILS and HILS that can utilize AI and big data.

1 is a block diagram illustrating a test environment for evaluating an autonomous driving module according to various embodiments of the present disclosure;
2 is a flowchart illustrating a method of evaluating an autonomous driving module in an evaluation system according to various embodiments of the present disclosure;
3 is a flowchart illustrating a method of transmitting data to an autonomous driving module for evaluation of an autonomous driving module in an evaluation system according to various embodiments of the present disclosure;
4 is a flowchart illustrating an operation of an autonomous driving module according to various embodiments of the present disclosure;

Hereinafter, various embodiments of the present document will be described with reference to the accompanying drawings. Examples and terms used therein are not intended to limit the technology described in this document to a specific embodiment, but it should be understood to include various modifications, equivalents, and/or substitutions of the embodiments. In connection with the description of the drawings, like reference numerals may be used for like components. The singular expression may include the plural expression unless the context clearly dictates otherwise. In this document, expressions such as “A or B” or “at least one of A and/or B” may include all possible combinations of items listed together. Expressions such as "first", "second", "first", or "second" can modify the corresponding elements regardless of order or importance, and can be used to distinguish one element from another element. However, the components are not limited. When an (eg, first) component is referred to as being “(functionally or communicatively) connected” or “connected” to another (eg, second) component, that component is It may be directly connected to the component or may be connected through another component (eg, a third component).

In this document, "configured (or configured to)" means "suitable for", "having the ability to", "modified to" depending on the context, for example, hardware or software. “,” “made to,” “capable of,” or “designed to,” may be used interchangeably. In some circumstances, the expression “a device configured to” may mean that the device is “capable of” with other devices or parts. For example, the phrase “a processor configured (or configured to perform) A, B, and C” refers to a dedicated processor (eg, an embedded processor) for performing the operations, or by executing one or more software programs stored in a memory device. , may refer to a general-purpose processor (eg, a CPU or an application processor) capable of performing corresponding operations.

1 is a block diagram illustrating a test environment for evaluating an autonomous driving module according to various embodiments of the present disclosure;

Referring to FIG. 1 , a test environment 100 for evaluating an autonomous driving module may be configured to include an evaluation system 101 , an autonomous driving software module 150 , and an autonomous driving hardware module 160 . However, the present invention is not limited thereto. For example, the test environment 100 may be configured to include only the evaluation system 101 and the autonomous driving software module 150 , or only the evaluation system 101 and the autonomous driving hardware module 160 .

According to various embodiments, the evaluation system 101 is connected to the autonomous driving software module 150 and the autonomous driving hardware module 160, and information related to the driving simulation is transmitted to the autonomous driving software module 150 and the autonomous driving hardware module ( 160), and evaluates and verifies the autonomous driving software module 150 and the autonomous driving hardware module 160 using information output from the autonomous driving software module 150 and the autonomous driving hardware module 160 can do. According to an embodiment, the evaluation system 101 may include a simulator 110 , a data generation module 120 , a verification module 130 , and an interface 140 .

According to various embodiments, the simulator 110 may reproduce the real-world driving environment and driving scenario on a virtual map, and may perform a driving simulation on the virtual vehicle according to the reproduced driving environment and driving scenario. According to an embodiment, the driving environment may include information about the surrounding environment of the virtual vehicle, such as weather, traffic volume, location of surrounding objects, location of pedestrians and pedestrian movement paths. According to an embodiment, the driving scenario may include information related to the movement of the virtual vehicle, such as a location, a driving direction, a driving speed, and/or a driving route of the virtual vehicle. According to an embodiment, the virtual map may be constructed as a 3D map. According to an embodiment, while the simulator 110 performs a driving simulation on the virtual vehicle, at least one virtual sensor (eg, a lidar sensor, a radar sensor, a wheel sensor, an image sensor, etc.) included in the virtual vehicle. The data generation module 120 , the autonomous driving software module 150 , and the autonomous driving generate data of the virtual vehicle and data related to the state of the virtual vehicle, and generate the data of the virtual sensor, the data associated with the state of the virtual vehicle, and the driving environment data. It may be provided by the hardware module 160 . According to an embodiment, when a control command received from the autonomous driving software module 150 or the autonomous driving hardware module 160 is received, the simulator 110 controls the virtual vehicle in the driving simulation according to the received control command. can For example, when a control command for stopping the vehicle is received from the autonomous driving software module 150 or the autonomous driving hardware module 160 while the virtual vehicle is driving in the driving simulation, the simulator 110 performs the driving simulation. The virtual vehicle can be stopped. According to an embodiment, the simulator 110 may include a display, and while the driving simulation is being performed, a driving simulation screen may be displayed through the display.

According to various embodiments, the data generating module 120 receives data of a virtual sensor, data related to a state of a virtual vehicle, and driving environment data from the simulator 110 while the simulator 110 performs a driving simulation, and , learning data for learning the autonomous driving software module 150 and/or the autonomous driving hardware module 160 based on the received data and the learning of the autonomous driving software module 150 and/or the autonomous driving hardware module 160 It is possible to generate evaluation data for evaluating the state. According to an embodiment, the evaluation data may be generated in various categories, such as cognitive evaluation data, judgment evaluation data, and control evaluation data. According to an embodiment, the cognitive evaluation data evaluates whether the autonomous driving software module 150 and/or the autonomous driving hardware module 160 accurately recognizes the driving environment (eg, pedestrians or objects) reproduced in the driving simulation. evaluation data can be presented. According to an embodiment, the judgment evaluation data evaluates whether the autonomous driving software module 150 and/or the autonomous driving hardware module 160 is making an appropriate situation judgment according to the driving environment and driving scenario reproduced in the driving simulation. data can be displayed. According to embodiments, the control evaluation data is used to evaluate whether the autonomous driving software module 150 and/or the autonomous driving hardware module 160 generates an appropriate control command according to the driving environment and driving scenario reproduced in the driving simulation. data can be displayed. When the learning data and the evaluation data are generated, the data generation module 120 may provide the generated learning data and the evaluation data to the autonomous driving software module 150 and/or the autonomous driving hardware module 160 .

According to various embodiments, the verification module 130 determines whether the control command performed by the autonomous driving software module 150 and/or the autonomous driving hardware module 160 based on data provided from the simulator 110 is an appropriate control command. It can be verified whether For example, the verification module 130 controls the autonomous driving software module 150 and/or the autonomous driving hardware module 160 when an obstacle (eg, a pedestrian or other vehicle) is located in front of the virtual vehicle in the driving simulation. By determining whether the virtual vehicle on the driving simulation has stopped (or avoided) so as not to collide with an obstacle according to the command, the control command provided from the autonomous driving software module 150 and/or the autonomous driving hardware module 160 is appropriately controlled It is possible to determine whether it is a command or not. As another example, when a red light is lit on a traffic light of a road on which the virtual vehicle is driven in the driving simulation, the verification module 130 controls the autonomous driving software module 150 and/or the autonomous driving hardware module 160 By determining whether the virtual vehicle in the driving simulation is stopped according to

According to various embodiments, the interface 140 may include an interface for supporting wired and/or wireless data communication between the simulator 110 , the autonomous driving hardware module 160 , and the verification module 130 . According to an embodiment, the interface 140 converts the data generated from the simulator 110 to correspond to the communication protocol supported by the autonomous driving hardware module 160 or the verification module 130 to the autonomous driving hardware module 160 . Alternatively, it may be provided to the verification module 130 . According to an embodiment, the interface 140 may convert data generated from the autonomous driving hardware module 160 to correspond to a communication protocol supported by the simulator 110 and provide it to the simulator 110 .

According to various embodiments, the autonomous driving software module 150 includes software that is programmed on a hardware device (eg, an electronic control unit (ECU)) capable of performing autonomous driving of a vehicle, and uses other software as necessary. may be modified to include According to an embodiment, the autonomous driving software module 150 generates a control command based on data provided from the simulator 110 (eg, data of a virtual sensor, data related to a state of a virtual vehicle, and driving environment data). and may provide the generated control command to the simulator 110 . According to an embodiment, the autonomous driving software module 150 performs learning based on the learning data and the evaluation data provided from the data generation module 120 and generates a control command in consideration of the learning result, so that the reliability is high. Control commands can be generated.

According to various embodiments, the autonomous driving hardware module 160 includes a hardware device (eg, an electronic control unit (ECU)) capable of performing autonomous driving of a vehicle, and includes other autonomous driving hardware devices as necessary. can be changed. According to an embodiment, the autonomous driving hardware module 160 transmits data (eg, virtual sensor data, virtual vehicle state-related data, and driving environment data) provided from the simulator 110 through the interface 140 . It may receive, generate a control command based on the received data, and provide the generated control command to the simulator 110 . According to an embodiment, the autonomous driving hardware module 160 performs learning based on the learning data and the evaluation data provided from the data generating module 120 and generates a control command in consideration of the learning result, so that the reliability is high. Control commands can be generated.

2 is a flowchart illustrating a method of evaluating an autonomous driving module in an evaluation system according to various embodiments of the present disclosure;

Referring to FIG. 2 , in operation 201 , an evaluation system (eg, the evaluation system 101 of FIG. 1 ) may perform a driving simulation. For example, the evaluation system 101 reproduces a real-world driving environment and driving scenario on a virtual map through the simulator 110 , and a driving simulation for a virtual vehicle according to the reproduced driving environment and driving scenario can be done According to an embodiment, the map for the driving simulation, the driving environment, and the driving scenario are set by a user of the evaluation system, and may be changed to other values as necessary.

In operation 203, the evaluation system 101 applies learning data and evaluation data generated based on data of the driving simulation while performing the driving simulation to the autonomous driving module (eg, the autonomous driving software module 150 of FIG. 1 and/or may be transmitted to the autonomous driving hardware module 160 of FIG. 1 . For example, while the simulator 110 performs a driving simulation, the data generation module 120 of the evaluation system 101 may generate data from a virtual sensor included in the virtual vehicle in the driving simulation from the simulator 110, data of the virtual vehicle, Receive data associated with the state and data associated with the driving environment, and generate learning data for learning the autonomous driving module and evaluation data for evaluating the learning state based on the received data, and the generated learning data and evaluation data can be transmitted to the autonomous driving module. According to an embodiment, while the driving simulation is being performed, the evaluation system 101 may provide the driving simulation data to the autonomous driving module in parallel (or sequentially) with the operation of generating the learning data and the evaluation data. . For example, while the driving simulation is being performed, the simulator 110 of the evaluation system 101 collects data of a virtual sensor included in the virtual vehicle in the driving simulation, data related to the state of the virtual vehicle, and data related to the driving environment. It can be transmitted to the autonomous driving module.

In operation 205 , the evaluation system 101 may verify the autonomous driving module. Specifically, the verification module 130 of the evaluation system 101 monitors the driving simulation reproduced through the simulator 110 , so that the autonomous driving module is based on the data provided from the simulator 110 and/or the data generation module 120 . It may be determined whether the control command generated based on the generated control command is a command suitable for the virtual vehicle in the driving simulation. For example, when an obstacle (eg, a pedestrian or other vehicle) is located in front of the virtual vehicle in the driving simulation, the verification module 130 may prevent the virtual vehicle in the driving simulation from collides with the obstacle according to a control command of the autonomous driving module. By determining whether the vehicle is stopped (or avoided), it can be determined whether the control command provided from the autonomous driving software module 150 and/or the autonomous driving hardware module 160 is an appropriate control command.

As described above, the evaluation system 101 generates data (eg, learning data and evaluation data) by which the autonomous driving module can learn based on the data generated through the driving simulation, and uses the generated data autonomously. By providing it as a driving module, it is possible to learn an AI-based autonomous driving module (eg, a hardware module or a software module), and evaluation and verification of an AI-based autonomous driving module that supports autonomous driving of a vehicle through driving simulation is performed. can be done

3 is a flowchart illustrating a method of transmitting data from an electronic device to an autonomous driving module for evaluation of the autonomous driving module according to various embodiments of the present disclosure; The following description may be a detailed operation of an operation of transmitting learning data and evaluation data to the autonomous driving module in operation 203 of FIG. 2 .

Referring to FIG. 3 , in operation 301 , the evaluation system (eg, the evaluation system 101 of FIG. 1 ) may transmit data of the driving simulation to the autonomous driving module. For example, while the driving simulation is performed in the simulator 110 , the evaluation system 101 autonomously analyzes data of a virtual sensor included in the virtual vehicle on the driving simulation, data associated with the state of the virtual vehicle, and data associated with the driving environment. may be transmitted to a driving module (eg, the autonomous driving software module 150 of FIG. 1 and/or the autonomous driving hardware module 160 of FIG. 1 ). According to an embodiment, the autonomous driving module generates a control command based on data provided from the simulator 110 (eg, data of a virtual sensor, data related to a state of a virtual vehicle, and driving environment data), and generates the generated control command. By providing the control command to the simulator 110 , it is possible to control the movement of the virtual vehicle in the driving simulation.

In operation 303 , the evaluation system 101 may generate learning data and evaluation data based on data of the driving simulation. For example, while the data generating module 120 of the evaluation system 101 performs the driving simulation in the simulator 110 , the data of the virtual sensor included in the virtual vehicle in the driving simulation from the simulator 110 and the state of the virtual vehicle It is possible to receive the data associated with the driving environment and the data associated with the driving environment, and generate learning data for learning the autonomous driving module and evaluation data for evaluating the learning state based on the received data.

In operation 305 , the evaluation system 101 may transmit the learning data and the evaluation data to the autonomous driving module in response to the learning data and the evaluation data being generated. For example, the data generation module 120 of the evaluation system 101 may directly provide the learning data and the evaluation data to the autonomous driving software module 150 . As another example, the data generation module 120 of the evaluation system 101 may provide learning data and evaluation data to the autonomous driving hardware module 160 through the interface 140 .

As described above, the evaluation system 101 may perform a process of transmitting driving simulation data to the autonomous driving module and a process of transmitting learning data and evaluation data to the autonomous driving module in parallel. In addition, when the autonomous driving module is implemented as hardware, the evaluation system 101 may support data communication with the autonomous driving module through the interface 140 that may support a communication protocol of the hardware.

4 is a flowchart illustrating an operation of an autonomous driving module according to various embodiments of the present disclosure.

Referring to FIG. 4 , in operation 401 , the autonomous driving module (eg, the autonomous driving software module 150 of FIG. 1 or the autonomous driving hardware module 160 of FIG. 1 ) is configured with an evaluation system (eg, the evaluation system of FIG. 1 ) 101)), learning can be performed based on the learning data and evaluation data provided. For example, the autonomous driving software module 150 may receive the learning data and the evaluation data generated from the data generating module 120 directly from the data generating module 120 , and learn the provided learning data and the evaluation data. As another example, the autonomous driving hardware module 160 may receive learning data and evaluation data generated from the data generation module 120 through the interface 140 , and learn the provided learning data and evaluation data.

In operation 403 , the autonomous driving module may receive data of the driving simulation. For example, the autonomous driving software module 150 may directly receive, from the simulator 110 , data of a virtual sensor included in the virtual vehicle in the driving simulation, data related to the state of the virtual vehicle, and driving environment data. As another example, the autonomous driving hardware module 160 may receive data of a virtual sensor included in the virtual vehicle in the driving simulation, data related to the state of the virtual vehicle, and driving environment data through the interface 140 .

In operation 403 , the autonomous driving module may transmit a control command corresponding to data of the driving simulation based on the learning. For example, the autonomous driving software module 150 generates a control command corresponding to the driving simulator based on learning, and directly provides the generated control command to the simulator 110 , thereby giving a control command to the virtual vehicle in the driving simulation. can reflect As another example, the autonomous driving hardware module 160 generates a control command corresponding to the driving simulator based on learning, and provides the generated control command to the simulator 110 through the interface 140 , so that Control commands can be reflected in the virtual vehicle.

In the above, it has been described that the autonomous driving module generates a control command corresponding to the data of the driving simulation after performing learning using the learning data and the evaluation data. However, the autonomous driving module is a learning process using the learning data and the evaluation data And, the process of generating a control command corresponding to the data of the driving simulation may be performed in parallel.

As used herein, the term “module” may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as, for example, logic, logic block, component, or circuit. A module may be an integrally formed part or a minimum unit or a part of the part that performs one or more functions. For example, according to an embodiment, the module may be implemented in the form of an application-specific integrated circuit (ASIC).

According to various embodiments, each component (eg, a module or a program) of the above-described components may include a singular or a plurality of entities. According to various embodiments, one or more components or operations among the above-described corresponding components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (eg, a module or a program) may be integrated into one component. In this case, the integrated component may perform one or more functions of each component of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. . According to various embodiments, operations performed by a module, program, or other component are executed sequentially, in parallel, repetitively, or heuristically, or one or more of the operations are executed in a different order, omitted, or , or one or more other operations may be added.

101 evaluation system 150 interface
120: simulator 160: autonomous driving software module
130: data generation module 170: autonomous driving hardware module
140: verification module

Claims (12)

a simulator that performs a driving simulation and generates data of the driving simulation;
a data generation module that generates learning data and evaluation data for learning the autonomous driving module based on the data of the driving simulation, and provides the learning data and the evaluation data to the autonomous driving module; and
a verification module for verifying the autonomous driving module;
The data generation module provides the learning data and the evaluation data directly to the autonomous driving software module of the autonomous driving module, and provides the learning data and the evaluation data to the autonomous driving hardware module through an interface to learn;
The simulator receives a control command corresponding to the data of the driving simulation from the autonomous driving software module, receives a control command from the autonomous driving hardware module through the interface, and performs the driving simulation,
While the driving simulation is being performed, a driving simulation screen is displayed through a display provided,
The verification module, as at least part of the operation of verifying the autonomous driving module, monitors the driving simulation performed in the simulator, and when a control command is received from the autonomous driving module, through the monitoring, the control command is An evaluation system that determines whether a control command is suitable for a virtual vehicle, and verifies the autonomous driving module based on the determination result.
According to claim 1,
The simulator, as at least part of the operation of generating data of the driving simulation,
While the driving simulation is being performed, the evaluation system generates data of a virtual sensor included in the virtual vehicle on the driving simulation, data related to a state of the virtual vehicle, and data related to a driving environment.
3. The method of claim 2,
The simulator is
An evaluation system for providing the data of the virtual sensor, the data related to the state of the virtual vehicle, and the data related to the driving environment to the data generating module and the autonomous driving module.
According to claim 1,
The simulator is
When the control command is received from the autonomous driving module, the evaluation system reflects the control command to the virtual vehicle in the driving simulation.
delete delete generating data of the driving simulation while the simulator of the evaluation system performs the driving simulation;
displaying, by the simulator, a driving simulation screen through a display included in the simulator while the driving simulation is performed;
generating, by the data generation module of the evaluation system, learning data and evaluation data for learning the autonomous driving module based on the data of the driving simulation;
providing, by the data generating module, the learning data and the evaluation data directly to the autonomous driving software module of the autonomous driving module, and providing the learning data and the evaluation data to the autonomous driving hardware module through an interface;
verifying, by the verification module of the evaluation system, the autonomous driving module; and
performing the driving simulation by receiving, by the simulator, a control command corresponding to the driving simulation data from the autonomous driving software module, and receiving a control command from the autonomous driving hardware module through the interface; including,
The step of verifying the autonomous driving module includes:
monitoring, by the verification module, the driving simulation performed in the simulator;
determining, by the verification module, whether the control command is a control command suitable for a virtual vehicle, through the monitoring, when a control command is received from the autonomous driving module; and
and verifying, by the verification module, the autonomous driving module based on a determination result.
8. The method of claim 7,
The step of generating data of the driving simulation comprises:
and generating, by the simulator, data of a virtual sensor included in the virtual vehicle on the driving simulation, data associated with the state of the virtual vehicle, and data associated with a driving environment while the driving simulation is being performed. how it works.
9. The method of claim 8,
and providing, by the simulator, data of the virtual sensor, data related to the state of the virtual vehicle, and data related to a driving environment to the data generating module and the autonomous driving module.
8. The method of claim 7,
and when the control command is received from the autonomous driving module by the simulator, reflecting the control command to the virtual vehicle in the driving simulation.
delete delete

Images