CN114174935A - Computer-implemented method and test unit for approximating a subset of test results - Google Patents
Computer-implemented method and test unit for approximating a subset of test results Download PDFInfo
- Publication number
- CN114174935A CN114174935A CN202080047817.8A CN202080047817A CN114174935A CN 114174935 A CN114174935 A CN 114174935A CN 202080047817 A CN202080047817 A CN 202080047817A CN 114174935 A CN114174935 A CN 114174935A
- Authority
- CN
- China
- Prior art keywords
- parameter set
- motor vehicle
- neural network
- artificial neural
- function
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B13/00—Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion
- G05B13/02—Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B13/00—Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion
- G05B13/02—Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric
- G05B13/0265—Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric the criterion being a learning criterion
- G05B13/027—Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric the criterion being a learning criterion using neural networks only
Landscapes
- Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Automation & Control Theory (AREA)
- Feedback Control In General (AREA)
- Traffic Control Systems (AREA)
Abstract
The invention relates to a computer-implemented method for approximating a subset of test results for a virtual test of a device for at least partially autonomously driving a motor vehicle. The invention further relates to a test unit (1) for approximating a subset of test results for a virtual test of a device for at least partially autonomously driving a motor vehicle. The invention also relates to a computer program and a computer-readable data carrier.
Description
Technical Field
The invention relates to a computer-implemented method for approximating a subset of test results for a virtual test of a device for at least partially autonomously driving a motor vehicle.
The invention further relates to a computer-implemented test unit for identifying a subset of test results for a virtual test of a device for at least partially autonomously driving a motor vehicle. The invention also relates to a computer program and a computer-readable data carrier.
Background
The driving assistance system, for example the adaptive speed controller and/or the functions for highly automated driving, can be verified or validated by means of various checking methods. In particular, hardware-in-the-loop methods, software-in-the-loop methods, simulation and/or test runs can be used here.
In this case, the effort, in particular the time and/or cost effort, for testing such vehicle functions using the above-described test method is typically very high, since a large number of potential driving situations must be tested.
This can lead to high costs for the test run and for the simulation in particular. DE 102017200180 a1 proposes a method for validating and/or verifying a vehicle function, which is provided for autonomously driving the vehicle in the longitudinal direction and/or the transverse direction.
The method comprises the following steps: a test control command for a vehicle function to a vehicle actuator is ascertained based on environmental data relating to the environment of the vehicle, wherein the test control command is not executed by the actuator.
The method further comprises: if the test control commands are executed, fictitious traffic situations that are to be present are simulated on the basis of the environmental data and using a traffic participant model relating to at least one traffic participant in the vehicle environment.
The method further comprises: test data is provided regarding fictitious traffic conditions. In order to ascertain test control commands in a vehicle, vehicle functions are operated passively.
This method has the disadvantage that the actual operation of the vehicle is required to ascertain the required data in order to verify and/or verify the vehicle function.
There is therefore a need for: the existing methods and test devices are improved in such a way that what are known as critical test cases in the context of context-based testing for systems and system components can be ascertained in an efficient manner during highly automated driving.
Disclosure of Invention
It is therefore an object of the present invention to provide a method, a test unit, a computer program and a computer-readable data carrier which enable critical test cases to be ascertained in an efficient manner in the context of a context-based test for systems and system components during highly automated driving.
This object is solved according to the invention by a computer-implemented method for approximately calculating a subset of test results for a virtual test of a device for at least partially autonomously driving a motor vehicle according to claim 1, a test unit for identifying a subset of test results for a virtual test of a device for at least partially autonomously driving a motor vehicle according to claim 12, a computer program according to claim 14 and a computer-readable data carrier according to claim 15.
The invention relates to a computer-implemented method for approximating a subset of test results for a virtual test of a device for at least partially autonomously driving a motor vehicle.
The method comprises the following steps: a data set defining a state space is provided, wherein each state is formed by a parameter set of driving situation parameters for which one or more actions can be carried out in order to realize a further parameter set from the parameter set, wherein each parameter set has at least one environmental parameter describing the environment of the motor vehicle and at least one own parameter describing the state of the motor vehicle.
The realization of the further parameter set in the state space is understood here to mean the finding of the further parameter set.
The method further comprises: an approximation calculation step is carried out, in which function values of at least one further parameter set are approximated using an artificial neural network, wherein the at least one further parameter set is identified as belonging to a subset of test results if the approximately calculated function values of the at least one further parameter set are greater than or equal to a predetermined threshold value.
If the function value of the at least one further parameter set is smaller than a predetermined threshold value, the artificial neural network carries out at least one further approximation calculation step starting from the respective last approximated further parameter set until the function value of the further parameter set is greater than or equal to the predetermined threshold value.
Within the scope of the method, an artificial neural network is therefore advantageously used, whose task is to approximate a subset of the test results. The subset of test results relates to critical test results of interest, which are to be the subject of a virtual test for a device, for example a control unit, which autonomously drives a motor vehicle.
In a scenario-based test of systems and system components for autonomously driving a motor vehicle, a scenario is defined, which may be referred to as an abstraction of a traffic situation. A logical scenario is here an abstraction of the traffic situation with roads, driving behavior and neighboring traffic without specific parameter values.
The specific context is obtained from the logical context by selecting a specific parameter value. Such specific scenarios correspond to respective individual traffic situations.
The autonomous driving function is implemented by a system, for example, a control unit. The control unit is tested in a conventional manner in real vehicles in real traffic situations or alternatively validated by virtual testing.
The method in this document calculates approximately critical test results or traffic situations, i.e. a subset of the entire test results that is considered critical. A critical test situation is, for example, all parameter combinations of particular driving situation parameters, which lead to critical driving situations, such as, for example, a vehicle collision or a quasi-vehicle collision.
In order to avoid unnecessarily large parameter combinations and traffic situations from being tested by conventional simulation methods, the above-mentioned subset of the test results which corresponds to the critical test situation is approximated by an artificial neural network within the scope of the method.
The test results thus calculated approximately can then be verified in an advantageous manner in the context of a virtual test of the control unit, so that a more effective virtual verification of the control unit for autonomously driving the motor vehicle can be achieved by the method according to the invention.
Thus by definition of the predetermined threshold value, according to the invention: when the test results of the approximation calculation can be identified as belonging to the desired subset.
The method is designed in such a way that it carries out an arbitrary number of approximation calculation steps until a relevant parameter set of the driving situation parameters is identified, which belongs to a subset of the test results of interest.
Further embodiments of the invention are the other dependent claims and the following description with reference to the drawings.
According to one aspect of the invention, the method according to the invention further comprises selecting an initial parameter set from a plurality of parameter sets of driving situation parameters, wherein the function value of each parameter set which can be realized by the initial parameter set through action is approximately calculated in the approximate calculation step by using an artificial neural network, wherein a selection step is implemented in which the parameter set which is approximately calculated in the approximate calculation step with the smallest or the highest function value is selected.
If the function value of the selected parameter set is smaller than a predetermined threshold value, at least one further selection step is carried out starting from the respective last selected parameter set until the function value of the selected parameter set is greater than or equal to the predetermined threshold value.
The method applies a reinforcement learning method for identifying critical test cases. In contrast to supervised learning in the case of the use of artificial neural networks, training is not carried out with the provided training data in reinforcement learning. Instead, there are two parties in reinforcement learning: networks, commonly referred to as agents; and the environment. The environment may also be considered a game floor (Spielfeld) where the current state or current location of the agent is read.
The network performs an action based on the current state. This action changes the state of the environment. The network in turn obtains the new state and the evaluation of the actions performed by the environment.
The aim is to achieve as good an evaluation as possible, i.e. to maximize the evaluation. Thus, the weights of the neural network are adapted and new actions are implemented in the learning process according to the evaluation of the carried out step (Zug). By a gradual adaptation of the weights, the network learns to implement the best possible actions, i.e. to obtain the best possible evaluated strategy. Similarly, if a minimum should be achieved, the evaluation can also be minimized by a simple adaptation.
One possible approach for identifying critical test cases is a Q-learning principle in which, starting from a certain state, all possible actions and their evaluations are taken into account. The action with the greatest benefit is selected and executed. For large states and motion spaces as in the present case, the Q function is implemented as a neural network. Such networks have the name DQN (deep Q network). The neural network approximates a Q function.
According to another aspect of the invention, the method according to the invention further comprises: the plurality of parameter sets of driving situation parameters is generated by means of an artificial neural network or by means of a simulation. The parameter set of the driving situation parameters can thus be generated in a simple manner, for example by applying a random function when using an artificial neural network, within a predefined definition domain.
According to a further aspect of the invention, the method according to the invention further comprises: the artificial neural network is provided with four hidden layers respectively comprising 128 neurons and an ELU activation function; and a coefficient y of 0.8 is used for attenuating the function value of the further approximation calculation step.
The approximate calculation of the critical test result is thus carried out in such a way that in each training step the artificial neural network is given as input to the current position and approximately calculates the function or Q value for the adjacent position.
The best neighbor location is ascertained from the highest revenue. The current position is changed to the best adjacent position. The training of the network thus comprises predicting the Q-values of neighbouring locations by the network according to the submitted locations. The adjacent position to which replacement is made is selected by means of the highest Q value. The Q value or function value at that location decays with a decay factor y.
To ascertain the gain, the direct gain of position is added to the attenuated Q value. As a theoretical value for the neural network, for determining the error and for updating the weights, a theoretical value of 0 is given, for example, for all positions except for the selected neighboring positions. The ascertained gain is given to the selected neighboring location.
According to a further aspect of the invention, the method according to the invention further comprises the step of selecting an initial parameter set from a plurality of parameter sets of the driving situation parameters, and if the function value of the further parameter set is smaller than a predetermined threshold value, the further artificial neural network evaluates the further parameter set approximately calculated by the artificial neural network.
The further artificial neural network is then adapted to the artificial neural network based on the evaluation. The artificial neural network adapted in this way performs at least one further approximation calculation step starting from the further parameter set of the respective last approximation calculation until the function value of the further parameter set is greater than or equal to a predetermined threshold value.
The above method relates to an Actor-commentary method (Actor-Critic-Verfahren). There are two parties in the actor-judge model: actors and judges. The actor obtains a state as in Q learning and performs an action according to the state. In contrast to Q-learning, no discretization is required in this method, so an action can be selected from a continuous number of actions. And also does not have to discretize the states.
The commentator evaluates the actions of the actor. To this end, the reviewer requires an assessment of the environment and a new state. The evaluator learns the evaluation of the predicted action by approximating the evaluation. The actor is adapted by the update of the commentator. The training of the actors and the judges is for example effected simultaneously.
The evaluation committee is given an evaluation of the status and environment, which is used as a theoretical value. The error can be calculated from the theoretical value and the actual value ascertained by the reviewer. The reviewer is updated by means of back-propagation. A feature in the training of actors is that the ascertained error to the judges is used to update the actor by means of back propagation.
According to another aspect of the invention, the method further comprises: the artificial neural network has four hidden layers respectively comprising 256 neurons and a PReLU activation function; the other artificial neural network is provided with four hidden layers respectively comprising 256 neurons and an ELU activation function; and the artificial neural network and the another artificial neural network apply an Adam optimization method.
According to another aspect of the invention, the method further comprises: the self-parameter comprises a speed of the motor vehicle, and the environment parameter comprises a speed of another motor vehicle and a distance between the motor vehicle and the other motor vehicle.
Using these parameters, for example, a so-called Cut-In scenario (Cut-In-Szenario) can be approximated. A cut-in scenario may be referred to as a traffic situation in which a highly automated or autonomous vehicle is traveling on a predetermined lane and another vehicle is driven into the lane of the host vehicle at a reduced speed compared to the host vehicle from the other lane at a predetermined spacing.
The speed of the own vehicle and the other vehicle, also called following vehicle, is here constant. Since the speed of the own vehicle is higher than that of the following vehicle, the own vehicle must be braked in order to avoid collision of the two vehicles.
Based on the above-mentioned own parameters and environmental parameters, critical traffic situations can therefore be approximated in a predefined domain of the above-mentioned parameters by the method according to the invention.
According to another aspect of the invention, the method further comprises: the function on which the function value is based is a safety objective function having a value for which the safety distance between the motor vehicle and the further motor vehicle is greater than or equal to VFELLOWX 0.55 and a maximum in the event of a collision between the motor vehicle and the further motor vehicle, and a safety spacing between the motor vehicle and the further motor vehicle is ≦ VFELLOWX 0.55 has a value greater than the minimum value.
The secure objective function states: the traffic situation is safe for the vehicle. The secure objective function is described in detail as follows: if the distance between the subject vehicle and the following vehicle is greater than or equal to the safe distance, the function value of the safe objective function is 0.
The safe distance may be defined as a distance at which safe braking of the host vehicle is always possible without a collision with the following vehicle, depending on the speed difference between the host vehicle and the following vehicle and the distance between the host vehicle and the following vehicle.
Such a distance is in the present example defined by a value in meters, which corresponds to the speed VFELLOW×0.55。
The objective function value approaches a value of 1 gradually as the distance between the host vehicle and the following vehicle is smaller or from below the safe distance. If a collision of the host vehicle with the following vehicle exists, the distance between the host vehicle and the following vehicle is therefore less than or equal to zero and the objective function value is 1.
According to a further aspect of the invention, the method according to the invention furthermore comprises: the function on which the function value is based is a comfort objective function or an energy consumption objective function having a value which has a minimum value in the case of no change in the acceleration of the motor vehicle and a maximum value in the case of a collision between the motor vehicle and the other motor vehicle, and which has a value between the minimum value and the maximum value depending on the amount of change in the acceleration in the case of a change in the acceleration of the motor vehicle.
With the aid of the comfort objective function, conclusions can be drawn as to how comfortable the driving maneuver is for the driver of the vehicle. Strong acceleration or braking and frequent repetition of these processes are considered uncomfortable.
The change in acceleration is called jerk (rock). The smaller the calculated value of the comfort objective function, the more comfortable the driving situation. The fuel consumption is 1 in the case of a collision of the own vehicle with a following vehicle, i.e. such that the fuel consumption is set to a determined maximum value. The reason for this is that the fuel tank of the vehicle can no longer be used further in the event of an accident.
With regard to the cut-in scenario, a possibly critical test case is therefore the boundary between collision and non-collision cases, which can be defined according to the respective objective functions, namely the safety objective function, the comfort objective function and the energy consumption objective function.
According to another aspect of the invention, the method comprises: a plurality of driving situation parameters, in particular the speed of the motor vehicle and the speed of the further motor vehicle, are generated by a stochastic algorithm within a predefined defined range. A plurality of driving situation parameters, which form a data set for approximating the computationally critical test results, can thus be generated in a simple and time-saving manner.
According to another aspect of the invention, the method further comprises: a separate artificial neural network is applied for approximating the value range of each function on which the function values are calculated, wherein the individual hyper-parameters of each artificial neural network are stored in a database.
Using a separate artificial neural network to approximate each individual objective function, i.e. the numerical range of the safety objective function, comfort objective function and/or energy consumption objective function, advantageously enables a more accurate approximate calculation result.
According to a further aspect of the invention, a test unit is provided for identifying a subset of test results for a virtual test of a device for at least partially autonomously driving a motor vehicle.
The test unit comprises means for providing a data set defining a state space, wherein each state is formed by a parameter set of driving situation parameters for which one or more actions can be carried out in order to realize a further parameter set from the parameter set, wherein each parameter set has at least one environmental parameter describing the environment of the motor vehicle and at least one own parameter describing the state of the motor vehicle.
The test unit further comprises an artificial neural network implementing an approximation calculation step in which function values of at least one further parameter set can be approximated, wherein the at least one further parameter set is identified as belonging to a subset of test results if the approximated function values of the at least one further parameter set are larger than or equal to a predetermined threshold value.
If the function value of the at least one further parameter set is smaller than a predetermined threshold value, the artificial neural network is configured to carry out at least one further approximation calculation step starting from the further parameter set which was calculated in each case as a last approximation until the function value of the further parameter set is greater than or equal to the predetermined threshold value.
In the context of the present test unit, an artificial neural network is therefore advantageously used, which has the task of approximately computing a subset of the test results, i.e. the critical test results of interest.
The test results thus approximately calculated can then advantageously be verified in the context of a virtual test of the control unit, so that a more effective virtual verification of the control unit for autonomously driven vehicles can be achieved by the test unit according to the invention.
According to a further aspect of the invention, the device is formed by a control unit, and the driving situation on which the approximate calculation of the test result of the virtual test of the control unit is based is a lane change of another motor vehicle to the lane of the motor vehicle using a plurality of driving situation parameters.
The test unit is thus advantageously able to approximate the respective test result of the virtual test with respect to, for example, a cut-in scenario.
According to a further aspect of the invention, a computer program is also specified, which comprises a program code for carrying out the method according to the invention when the computer program is executed on a computer. According to a further aspect of the invention, a data carrier is provided, which comprises a program code of a computer program for carrying out the method according to the invention when the computer program is executed on a computer.
The features of the method described herein may be used to approximate the critical test results for a number of different scenarios or driving situations. The test unit according to the invention is likewise suitable for testing a plurality of different devices or control units, for example of automobiles, trucks and/or commercial vehicles, ships or aircraft, in the sense of critical test results.
Drawings
For a better understanding of the present invention and its advantages, reference is now made to the following descriptions taken in conjunction with the accompanying drawings. The invention is further elucidated below on the basis of exemplary embodiments, which are given in the respective schematic drawings of the drawing. The figures show:
fig. 1 shows a flow chart of a method for approximating a subset of test results for a virtual test of a device for at least partially autonomously driving a motor vehicle according to a preferred embodiment of the invention;
FIG. 2 shows a schematic diagram of an approximation calculation method according to the present invention, according to a preferred embodiment of the present invention;
FIG. 3 shows a schematic diagram of an approximation calculation method according to the present invention, according to a preferred embodiment of the present invention;
fig. 4 shows a flow diagram of a DQN network according to the invention, according to a preferred embodiment of the invention;
fig. 5 shows a flow chart of a method for approximating a subset of test results for a virtual test of a device for at least partially autonomously driving a motor vehicle according to a further preferred embodiment of the invention;
FIG. 6 shows a further flowchart of the method shown in FIG. 5 according to a further preferred embodiment of the invention;
FIG. 7 shows a 3-dimensional diagram of an objective function according to the invention in accordance with a further preferred embodiment of the invention;
FIG. 8 shows a 2-dimensional representation of a cross section of the objective function according to the invention shown in FIG. 7 according to a further preferred embodiment of the invention; and
fig. 9 shows a 2-dimensional representation of a cross section of the objective function according to the invention shown in fig. 7 according to a further preferred embodiment of the invention.
Like reference numerals refer to like elements of the drawings unless otherwise specified.
Detailed Description
Fig. 1 shows a flow chart of a method for approximating a subset of test results for a virtual test of a device for at least partially autonomously driving a motor vehicle according to a preferred embodiment of the invention.
The method comprises providing S1 a data set D defining a state space Z. Each state Z1, Z2 … Zn is formed by a parameter set P1, P2 … Pn of driving situation parameters. One or more actions can be performed for the respective state Z1, Z2 … Zn in order to implement a further parameter set P1, P2 … Pn from the parameter sets P1, P2 … Pn, wherein each parameter set P1, P2 … Pn has at least one environmental parameter describing the vehicle environment and at least one own parameter describing the vehicle state.
The method further comprises an approximate calculation step S2, in which function values F1, F2 … Fn of at least one further parameter set P1, P2 … Pn are approximately calculated using an artificial neural network K1.
Identifying (S3) the at least one further parameter group P1, P2 … Pn as belonging to the subset of test results if the function value F1, F2 … Fn of the approximately calculated at least one further parameter group P1, P2 … Pn is greater than or equal to a predetermined threshold value W. If the function values F1, F2 … Fn of the at least one further parameter set P1, P2 … Pn are smaller than the predetermined threshold value W, the artificial neural network K1 carries out at least one further approximation calculation step S4 starting from the respective last approximated further parameter set P1, P2 … Pn until the function values F1, F2 … Fn of the at least one further parameter set P1, P2 … Pn are greater than or equal to the predetermined threshold value W.
A plurality of parameter sets (P1, P2 … Pn) of driving situation parameters are generated by the artificial neural network K1. Alternatively, the plurality of parameter sets (P1, P2 … Pn) may be generated, for example, by simulation.
In the present embodiment, the artificial neural network K1 has four hidden layers each comprising 128 neurons and an ELU activation function. The function values F1, F2 … Fn of the further approximation calculation step are further attenuated with a coefficient γ of 0.8.
Fig. 2 shows a schematic representation of the approximation calculation method according to the invention in accordance with a preferred embodiment of the invention.
Fig. 2 is a diagram showing a Q learning game field composed of 10 × 10 squares. The target point is located in the middle of the game field and is marked in black. In the context of the present Q learning method, the respective objective function for a given situation or traffic situation to be tested is approximately calculated using a neural network, in particular a DQN (deep Q network).
The initial position is specified at random. The action moves the current location to an adjacent location. Accordingly, the transition from one position to the adjacent upper, right, lower or left grid is possible. As a direct benefit, a predetermined value, e.g., 100, is specified for the grid target grid, while another value, e.g., 0, is determined for each other grid in the field.
During the game (Spieldurchlauf), the position is changed from the initial position to the new position for as long as the target position is reached. Followed by a new game play starting from a new random initial position. The training of the neural network for approximating the Q function ends after a predetermined number of game sessions, for example 1000 game sessions. The value of the coefficient γ for the attenuation gain is, for example, set to 0.8.
Fig. 3 shows a schematic representation of the approximation calculation method according to the invention in accordance with a preferred embodiment of the invention.
Fig. 3 shows an exemplary use of the driving situation parameter VEGOI.e. speed of the vehicle and V on the vertical axisFELLOWI.e. a cut-in scenario of the speed of the following vehicle travelling ahead.
The function shown in fig. 3 forms the boundary between critical and non-critical test results and substantially corresponds to the objective function shown in fig. 2. The points shown are the test results of the approximation calculation. Alternatively, the points shown may be, for example, simulated test results.
The function shown relates to a safety objective function having a value of the safety between the motor vehicle and the further motor vehicleFull spacing is more than or equal to VFELLOWX 0.55 and a maximum in the event of a collision between the motor vehicle and the further motor vehicle, and a safety spacing between the motor vehicle and the further motor vehicle is ≦ VFELLOWX 0.55 has a value greater than the minimum value.
Alternatively to the safety objective function, for example, a comfort objective function or an energy consumption objective function may be approximated, which has a value that has a minimum value in the case of no change in the acceleration of the motor vehicle and a maximum value in the case of a collision between the motor vehicle and the further motor vehicle, and that has a value between the minimum value and the maximum value depending on the amount of change in the acceleration in the case of a change in the acceleration of the motor vehicle.
Multiple driving situation parameters, in particular the speed V of the motor vehicleEGOAnd the speed V of the other motor vehicleFELLOWGenerated by a random algorithm within a predetermined defined field. Alternatively, the plurality of driving situation parameters can be generated, for example, by simulation.
A separate artificial neural network is applied for approximating the numerical range of each function on which the function values are calculated. The individual hyper-parameters of each artificial neural network are stored in a database.
Fig. 4 shows a flow diagram of a DQN network according to the invention, according to a preferred embodiment of the invention.
An initial parameter set SP is selected from the plurality of parameter sets P1, P2 … Pn of driving situation parameters. In An approximation calculation step S2, function values F1, F2 … Fn for each of the adjacent parameter groups P1, P2 … Pn realizable by the initial parameter group SP through actions a1, a2 … An are approximately calculated using the artificial neural network K1.
Next, a selection step S2A is carried out, in which parameter sets P1, P2 … Pn are selected, which were approximately calculated in the approximation calculation step with the smallest or the highest function values F1, F2 … Fn.
If the function values F1, F2 … Fn of the selected parameter set are smaller than the predetermined threshold value W, at least one further selection step S2B is carried out starting from the respective last selected parameter set P1, P2 … Pn until the function values F1, F2 … Fn of the selected parameter set are greater than or equal to the predetermined threshold value W.
Fig. 5 shows a flow chart of a method for approximating a subset of test results for a virtual test of a device for at least partially autonomously driving a motor vehicle according to a further preferred embodiment of the invention.
Another alternative embodiment of the present invention relates to an actor-critic method or model in comparison to the Q learning method. In the actor-judge approach, discretization of state and action is not necessary. For applications such as cut-in scenarios, the state is VEGOAnd VFELLOWAnd (4) value pairs. These are shown in fig. 5, for example, as parameter sets P1, P2.
From the determined parameter set, a mapping to each of the other arbitrary parameter sets is possible. It is therefore not necessary to convert to a determined pair of neighborhood values or set of neighborhood parameters, as is the case in Q learning. The stride is arbitrary and may implement a value pair that may not be achievable based on discretization in Q learning.
Two application scenarios are considered as in Q learning. Critical test cases in which a collision occurs or which lie on the boundary between collision and non-collision cases should be identified. As an evaluation by the environment, for example, a security objective function is applied.
An initial parameter set SP is selected from the plurality of parameter sets P1, P2 … Pn of driving situation parameters.
If the function values F1, F2 … Fn of the further parameter set P1, P2 … Pn are smaller than the predetermined threshold value W, the further artificial neural network K2 evaluates the further parameter set P1, P2 … Pn approximately calculated by the artificial neural network K1 and adapts the artificial neural network K1 in step S5 on the basis of the evaluation BW.
The artificial neural network K1 adapted in this way performs at least one further approximation calculation step S4 starting from the further parameter set P1, P2 … Pn calculated in each case as a last approximation until the function values F1, F2 … Fn of the further parameter set P1, P2 … Pn are greater than or equal to the predetermined threshold value W.
Training or learning of the second artificial neural network K2 or the panel network is effected in step S6. Training of the panelist network is achieved by means of back propagation. The evaluation network or another neural network K2 is given status and an evaluation of the environment, which is used as a theoretical value. The error can be calculated from the theoretical value and the actual value ascertained by the network of judges. Subsequently, the network of judges is updated by means of back propagation.
Fig. 6 shows a further flowchart of the method shown in fig. 5 according to a further preferred embodiment of the invention.
The artificial neural network K1 has four hidden layers each including 256 neurons and a prilu activation function. The further artificial neural network K2 has four hidden layers each comprising 256 neurons and an ELU activation function. The artificial neural network K1 and the further artificial neural network K2 apply Adam optimization methods.
The self-parameter FP3 comprises the speed V of the motor vehicleEGO. The environmental parameters FP1, FP2 comprise the speed V of the further motor vehicleFELLOWAnd a distance d between the motor vehicle and the further motor vehicleSPUP。
The artificial neural network K1 receives the value pair VEGOAnd VFELLOWAnd a spacing dSPUPAs input variables and to value VEGOAnd VFELLOWConversion to New value pair V'EGOAnd V'FELLOW。
The further artificial neural network K2 evaluates the new value pair V'EGOAnd V'FELLOW. The adaptation of the artificial neural network K1 is effected by means of the evaluation of the further artificial neural network K2.
Fig. 7 shows a 3-dimensional diagram of an objective function according to the invention in accordance with a further preferred embodiment of the invention.
The function shown is a truncated cone with a constant peak. The purpose of the approximation calculation method is to achieve points lying on the plane of the cone. The parameter pairs P1, P2 are assigned to the respective function to be determined. These parameter pairs may be, for example, the speed V of the own vehicleEGOAnd following the speed V of the vehicleFELLOW。
The further artificial neural network K2 has been previously or previously trained. The neural network or the actor network is updated on the basis of the evaluation of the further neural network or the critic network in order to achieve as good a point as possible. The evaluation of the network of judges is an actual value and the theoretical value is the maximum value of the function.
For training, a random initial position is generated similarly to in Q learning. The aim is to shift this initial position into the target region, i.e. into the plane of the cone. The initial position is submitted to the actor network. The actor network converts the initial position to a new position.
The actor network is updated based on the new location through the review network's evaluation. The current location is then again transitioned from the actor network to the new location. This process is repeated a number of times. The actor network is thus updated at each step according to the ratings of the commentator network.
Fig. 8 shows a 2-dimensional representation of a cross section of the objective function according to the invention shown in fig. 7 according to a further preferred embodiment of the invention.
The circular surface shown in fig. 8 contains or corresponds to the target region of the function. The points shown are test results calculated approximately by the method according to the invention.
Fig. 9 shows a 2-dimensional representation of a cross section of the objective function according to the invention shown in fig. 7 according to a further preferred embodiment of the invention.
In this function, the target region is defined relatively narrowly and corresponds to the edge region of the target region shown in fig. 8. The points arranged along the edge region of the line shape correspond to the test results approximately calculated by the method.
As can be seen from fig. 9, the approximately calculated test results are located in a given target region and thus correspond to a subset of the test results, i.e. the critical test results of interest.
Fig. 1 and 5 also show a test unit 1 according to the invention for identifying a subset of test results of a virtual test for a device for at least partially autonomously driving a motor vehicle. The test unit 1 comprises a corresponding device 2 for providing a data set D defining a state space Z and an artificial neural network K1 and/or a further artificial neural network K2.
Although specific embodiments have been illustrated and described herein, it will be appreciated by those of ordinary skill in the art that a variety of alternate and/or equivalent implementations exist. It should be noted that the exemplary embodiment or exemplary embodiments are only examples and are not intended to limit the scope, applicability, or configuration in any way.
Moreover, the foregoing summary and detailed description will provide those skilled in the art with a convenient road map for implementing at least one exemplary embodiment, wherein it is apparent that various changes may be made in the functional scope and arrangement of elements without departing from the scope of the appended claims and their legal equivalents.
In general, this application is intended to cover any adaptations or variations of the embodiments discussed herein.
Claims (15)
1. A computer-implemented method for approximating a subset of test results for a virtual test of a device for at least partially autonomously driving a motor vehicle, the method comprising the steps of:
providing (S1) a data set (D) defining a state space (Z), wherein each state (Z1, Z2 … Zn) is formed by a parameter set (P1, P2 … Pn) of driving situation parameters, for which state (Z1, Z2 … Zn) one or more actions (A1, A2 … An) can be carried out in order to realize a further parameter set (P1, P2 … Pn) from the parameter sets (P1, P2 … Pn), wherein each parameter set (P1, P2 … Pn) has at least one environmental parameter (FP1, FP2) describing the vehicle environment and at least one self parameter (FP3) describing the vehicle state;
-carrying out an approximate calculation step (S2) in which the function values (F1, F2 … Fn) of at least one further parameter set (P1, P2 … Pn) are approximately calculated using an artificial neural network (K1), and if the approximately calculated function values (F1, F2 … Fn) of the at least one further parameter set (P1, P2 … Pn) are greater than or equal to a predetermined threshold value (W), the at least one further parameter set (P1, P2 … Pn) is identified (S3) as belonging to a subset of test results; if the function values (F1, F2 … Fn) of the at least one further parameter set (P1, P2 … Pn) are smaller than a predetermined threshold value (W), the artificial neural network (K1) performs at least one further approximation calculation step (S4) starting from the further parameter set (P1, P2 … Pn) of the respective last approximation calculation until the function values (F1, F2 … Fn) of the further parameter set (P1, P2 … Pn) are greater than or equal to the predetermined threshold value (W).
2. The computer-implemented method according to claim 1, characterized in that one initial parameter Set (SP) is selected (S1) from a plurality of parameter sets (P1, P2 … Pn) of driving situation parameters, wherein function values (F1, F2 … Fn) of each neighboring parameter set (P1, P2 … Pn) that can be realized by the initial parameter Set (SP) by means of actions (a1, a2 … An) are approximately calculated using An artificial neural network (K1) in An approximate calculation step (S2), wherein a selection step (S2A) is implemented in which a parameter set (P1, P2 … Pn) is selected for which the smallest or highest function value (F1, F2 … Fn) is approximately calculated in the approximate calculation step (S2), and if the last parameter set (P1) of the selected parameter set (P1, P48 Pn) is smaller than a predetermined threshold value (W1), the last parameter set (P58573) is selected, P2 … Pn) is initiated, at least one further selection step (S2B) is carried out until the function values (F1, F2 … Fn) of the selected parameter set (P1, P2 … Pn) are greater than or equal to a predetermined threshold value (W).
3. The computer-implemented method according to claim 1 or 2, characterized in that a plurality of parameter sets (P1, P2 … Pn) of driving situation parameters are generated by the artificial neural network (K1) or by simulation.
4. The computer-implemented method according to one of the preceding claims, characterized in that the artificial neural network (K1) has four hidden layers each comprising 128 neurons and an ELU activation function; and attenuating the function value of the further approximation calculation step (F1, F2 … Fn) with a coefficient γ of 0.8.
5. The computer-implemented method according to claim 1, characterized in that an initial parameter Set (SP) is selected from a plurality of parameter sets (P1, P2 … Pn) of driving situation parameters, if the function values (F1, F2 … Fn) of the further parameter set (P1, P2 … Pn) are smaller than a predetermined threshold value (W), a further artificial neural network (K2) evaluates the further parameter set (P1, P2 … Pn) approximately calculated by the artificial neural network (K1) and adapts the artificial neural network (K1) on the basis of the evaluation (BW), and the artificial neural network (K1) adapted in this way carries out at least one further approximation calculation step (S3) starting from the further parameter set (P1, P2 … Pn) calculated in each case as a last approximation, until the function values (F1, F2 … Fn) of the further parameter set (P1, P2 … Pn) are greater than or equal to a predetermined threshold value (W).
6. The computer-implemented method according to claim 5, characterized in that the artificial neural network (K1) has four hidden layers each comprising 256 neurons and has one PReLU activation function; the further artificial neural network (K2) has four hidden layers each comprising 256 neurons and has an ELU activation function; and the artificial neural network (K1) and the further artificial neural network (K2) apply an Adam optimization method.
7. The computer-implemented method according to one of the preceding claims, characterized in that the own parameter (FP3) comprises a speed (V) of the motor vehicleEGO) And the environmental parameter (FP1, FP2) comprises the speed (V) of the other motor vehicleFELLOW) And a distance (d) between the motor vehicle and the further motor vehicleSPUR)。
8. The computer-implemented method of one of the preceding claims, wherein the function on which the function values (F1, F2 … Fn) are based is a safety objective function having a value for a safety distance between the motor vehicle and the other motor vehicle≥VFELLOWX 0.55 and a maximum in the event of a collision between the motor vehicle and the further motor vehicle, and a safety spacing between the motor vehicle and the further motor vehicle is ≦ VFELLOWX 0.55 has a value greater than the minimum value.
9. The computer-implemented method according to one of claims 1 to 7, characterized in that the function on which the function value (F1, F2 … Fn) is based is a comfort objective function or an energy consumption objective function, which comfort objective function or energy consumption objective function has a value which has a minimum value in the case of no change in acceleration of the motor vehicle and a maximum value in the case of a collision between the motor vehicle and the other motor vehicle, and which has a value between the minimum value and the maximum value in accordance with the amount of change in acceleration in the case of a change in acceleration of the motor vehicle.
10. Computer-implemented method according to one of claims 7 to 9, characterized in that the plurality of driving situation parameters, in particular the speed (V) of the motor vehicleEGO) And the speed (V) of the other motor vehicleFELLOW) Generated by a random algorithm within a predetermined defined field.
11. The computer-implemented method according to one of claims 8 to 10, characterized in that a separate artificial neural network is applied for each function's value range on which the function values (F1, F2 … Fn) are approximately calculated, wherein the individual hyper-parameters of each artificial neural network are stored in a database.
12. Test unit (1) for approximating a subset of test results for a virtual test of a device for at least partially autonomously driving a motor vehicle, the test unit comprising:
means (2) for providing a data set (D) defining a state space (Z), wherein each state (Z1, Z2 … Zn) is formed by a parameter set (P1, P2 … Pn) of driving situation parameters, for which state one or more actions (A1, A2 … An) can be carried out in order to realize a further parameter set (P1, P2 … Pn) from the parameter sets (P1, P2 … Pn), wherein each parameter set (P1, P2 … Pn) has at least one environmental parameter (FP1, FP2) describing the vehicle environment and at least one own parameter (FP3) describing the vehicle state;
an artificial neural network (K1) which implements an approximation calculation step in which function values (F1, F2 … Fn) of at least one further parameter set (P1, P2 … Pn) can be approximated, which at least one further parameter set (P1, P2 … Pn) can be identified as belonging to a subset of test results if the function values (F1, F2 … Fn) of the at least one further parameter set (P1, P2 … Pn) which have been approximated are greater than or equal to a predetermined threshold value (W); if the function values (F1, F2 … Fn) of the at least one further parameter set (P1, P2 … Pn) are smaller than a predetermined threshold value (W), the artificial neural network (K1) is configured to carry out at least one further approximation calculation step starting from the respective last approximately calculated further parameter set (P1, P2 … Pn) until the function values (F1, F2 … Fn) of the further parameter set (P1, P2 … Pn) are greater than or equal to the predetermined threshold value (W).
13. The test unit according to claim 12, characterized in that the device is formed by a control unit and the driving situation on which the approximate calculation of the test result of the virtual test of the control unit is based is a lane change of another motor vehicle to the lane of the motor vehicle using a plurality of driving situation parameters.
14. Computer program comprising a program code for performing the method according to one of claims 1 to 11 when the computer program is executed on a computer.
15. Computer-readable data carrier, which comprises a program code of a computer program for carrying out the method according to one of claims 1 to 11 when the computer program is executed on a computer.
Applications Claiming Priority (3)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| EP19192741.7 | 2019-08-21 | ||
| EP19192741.7A EP3783446B1 (en) | 2019-08-21 | 2019-08-21 | Computer-implemented method and test unit for approximating a subset of test results |
| PCT/EP2020/073062 WO2021032715A1 (en) | 2019-08-21 | 2020-08-18 | Computer implemented method and test unit for approximating a subset of test results |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114174935A true CN114174935A (en) | 2022-03-11 |
Family
ID=67659615
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202080047817.8A Pending CN114174935A (en) | 2019-08-21 | 2020-08-18 | Computer-implemented method and test unit for approximating a subset of test results |
Country Status (3)
| Country | Link |
|---|---|
| EP (1) | EP3783446B1 (en) |
| CN (1) | CN114174935A (en) |
| WO (1) | WO2021032715A1 (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112997128B (en) * | 2021-04-19 | 2022-08-26 | 华为技术有限公司 | Method, device and system for generating automatic driving scene |
| EP4199553A1 (en) * | 2021-12-14 | 2023-06-21 | dSPACE GmbH | Method and test unit for test execution of virtual tests |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| DE102016208076A1 (en) * | 2016-05-11 | 2017-11-16 | Continental Teves Ag & Co. Ohg | METHOD AND DEVICE FOR EVALUATING AN INPUT VALUE IN A DRIVER ASSISTANCE SYSTEM, DRIVER ASSISTANCE SYSTEM AND TEST SYSTEM FOR A DRIVER ASSISTANCE SYSTEM |
| DE102017200180A1 (en) | 2017-01-09 | 2018-07-12 | Bayerische Motoren Werke Aktiengesellschaft | Method and test unit for the motion prediction of road users in a passively operated vehicle function |
-
2019
- 2019-08-21 EP EP19192741.7A patent/EP3783446B1/en active Active
-
2020
- 2020-08-18 CN CN202080047817.8A patent/CN114174935A/en active Pending
- 2020-08-18 WO PCT/EP2020/073062 patent/WO2021032715A1/en active Application Filing
Also Published As
| Publication number | Publication date |
|---|---|
| EP3783446B1 (en) | 2021-08-11 |
| WO2021032715A1 (en) | 2021-02-25 |
| EP3783446A1 (en) | 2021-02-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR102461831B1 (en) | System and Method for Improving of Advanced Deep Reinforcement Learning Based Traffic in Non signalalized Intersections for the Multiple Self driving Vehicles | |
| Krasowski et al. | Safe reinforcement learning for autonomous lane changing using set-based prediction | |
| Li et al. | Game theoretic modeling of driver and vehicle interactions for verification and validation of autonomous vehicle control systems | |
| CN108698595B (en) | For controlling the method for vehicle movement and the control system of vehicle | |
| Gelenbe et al. | Simulation with learning agents | |
| Nishi et al. | Merging in congested freeway traffic using multipolicy decision making and passive actor-critic learning | |
| Brechtel et al. | Probabilistic MDP-behavior planning for cars | |
| CN111679660B (en) | Unmanned deep reinforcement learning method integrating human-like driving behaviors | |
| CN111301419A (en) | Reinforcement learning based method for SAE4 level automated lane change | |
| CN110686906B (en) | Automatic driving test method and device for vehicle | |
| González et al. | High-speed highway scene prediction based on driver models learned from demonstrations | |
| CN114174935A (en) | Computer-implemented method and test unit for approximating a subset of test results | |
| US20190392308A1 (en) | Grading And Unlearning Implementations For Neural Network Based Course Of Action Selection | |
| CN111079800B (en) | Acceleration method and acceleration system for intelligent driving virtual test | |
| CN112382165B (en) | Driving strategy generation method, device, medium, equipment and simulation system | |
| Li et al. | An explicit decision tree approach for automated driving | |
| CN113511222A (en) | Scene self-adaptive vehicle interactive behavior decision and prediction method and device | |
| WO2022100835A1 (en) | Computing system and method for trajectory planning in a simulation road driving environment | |
| CN113110359B (en) | Online training method and device for constraint type intelligent automobile autonomous decision system | |
| Venkatesh et al. | Connected and automated vehicles in mixed-traffic: Learning human driver behavior for effective on-ramp merging | |
| US20220138094A1 (en) | Computer-implemented method and test unit for approximating a subset of test results | |
| CN114987511A (en) | Method for simulating human driving behavior to train neural network-based motion controller | |
| Lienke et al. | Core components of automated driving–algorithms for situation analysis, decision-making, and trajectory planning | |
| Barbier et al. | Probabilistic decision-making at road intersections: Formulation and quantitative evaluation | |
| CN116127853A (en) | Unmanned driving overtaking decision method based on DDPG (distributed data base) with time sequence information fused |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |