CN111752261A - Autonomous driving test platform based on autonomous driving robot - Google Patents

Abstract

The present invention provides an automatic driving test platform based on an autonomous driving robot, which is characterized by comprising: a car driving simulator with a driver's seat, a display screen and a simulated driving mechanism arranged in front of the driver's seat, and a computer respectively connected in communication with the simulated driving mechanism; and a driving robot, which sits on the driving seat and operates the simulated driving mechanism, has a robot body matched with the driving seat, and is arranged on the robot body A monocular camera and a driving operating mechanism on the device, as well as a central controller connected in communication with the binocular camera, the driving operating mechanism and the computer, respectively. The autonomous driving test platform is low-cost and can meet various application scenarios required in driverless vehicle testing.

Description

Autonomous driving test platform based on autonomous driving robot

technical field

The invention belongs to the field of unmanned driving, relates to the optimization and testing of unmanned driving algorithms, and in particular relates to an automatic driving test platform based on an autonomous driving robot.

Background technique

In the field of unmanned vehicles, every step of the development of technology must be measured by ensuring personal safety. At present, there are two bottlenecks in the development of unmanned vehicles: one is the official requirements for the minimum test mileage of vehicles, and the other is driving decision-making The model requires massive data in different driving scenarios for testing and verification. Studies have shown that, in order to achieve the safety level of human-like drivers, the safety of unmanned driving needs billions of miles of experimental miles to prove it. However, even with the most reasonable intentions, existing driverless cars will take decades, if not hundreds of years, to complete the intended range test. Therefore, if the test is put on the real road, it will be an impossible task in the short term. In general, road testing is expensive and time-consuming, and virtual testing is the only way to go, but its virtual environment and the validity of test results need to be demonstrated and improved.

Simulation from software to hardware, when properly modeled, opens up possibilities for companies to test and validate their vehicle models. This includes a variety of application scenarios, including traffic information, driver behavior, weather, and road conditions. Google, Tesla, Zoox…and many more are using simulation to try to get self-driving cars to a billion miles as quickly as possible. Today, products such as Vires, TaSS, PreScan, CarSim, Oktal, ScanNer, and ROSGazebo offer engineers the possibility to simulate sensors, their generation mechanisms, and mechanical structures. Despite their strengths, they simultaneously overlook areas critical to simulation, including oversimplification of existing sensor outputs and an understanding of how the environment affects the sophistication of autonomous models. In addition, this kind of driving scene simulation software is not only expensive, but also the simulated scene is quite different from the real environment, which makes it difficult to simulate the perception of the outside world by most sensors. Once there is a problem in the simulation process, it will lead to problems such as traffic accidents that are easily endangered to life and safety.

As above, compared with traditional cars, due to the complexity of the system of driverless cars, in addition to the laboratory simulation related to traditional cars and the test tests of conventional car proving grounds, the vehicle also needs to carry out a large number of roads in various scenarios. The test requires training and learning its autonomous driving ability to meet the safety requirements. At present, the technical means of testing driverless vehicles mainly include intelligent networked vehicle proving ground and virtual simulation. At present, many countries in the world have begun to build relevant ICV test sites to serve the development of ICVs. At present, foreign ICV test sites in operation include: Ann Arbor Demonstration Area (M-City) in the United States, Silicon Valley Demonstration Area (WillowRun) in the United States, ITS Corridor in Europe, AstaZero Test Site in Sweden, Tsukuba Science City in Japan, etc. The construction cost of a collaborative intelligent network-connected vehicle test field that can meet the communication test environment of all working conditions is usually more than hundreds of millions of yuan. In addition to the shortcomings of insufficient environmental restoration, such as the above-mentioned PreScan and other genuine virtual simulation test software, the license fee of the software is also more than one million.

SUMMARY OF THE INVENTION

In order to solve the above problems, a kind of automatic driving simulation platform with low cost and high fidelity test environment for various application scenarios required in vehicle testing is provided, and the present invention adopts the following technical solutions:

The present invention provides an automatic driving test platform based on an autonomous driving robot, which is characterized by comprising: a car driving simulator, which has a driver seat, a display screen and a simulated driving mechanism arranged in front of the driver seat, and a display screen and a driving mechanism. A computer that is respectively connected to the simulated driving mechanism; and a driving robot, which sits on the driver's seat and operates the simulated driving mechanism, and has a robot body matched with the driver's seat, a binocular camera and a driving operating mechanism arranged on the robot body. , and a central controller that communicates with the camera, the driving operating mechanism and the computer respectively, wherein the computer has a simulation engine storage part and a driving simulation screen generation control part, and the central controller has a driving model storage part, a control instruction generation part, a driving The control part, the driving simulation data acquisition and storage part, the decision model iteration part, and the controller communication part, the simulation engine storage part stores a variety of high-simulation racing game engines, and the driving model storage part stores the information used to generate the decision-making for the driving operation of the vehicle. The driving decision model of the driving decision information and the vehicle control model used to generate the corresponding robot control instructions according to the driving decision. The driving simulation screen generation control unit generates the driving scene image that simulates the virtual driving environment based on the high-simulation racing game engine and controls the display screen. For corresponding display, the binocular camera captures the driving scene image displayed on the display screen in real time and outputs a pair of screen capture images obtained by binocular capture in real time to the central controller. Once the controller communication part receives the screen capture image, the control instruction The generation part generates the corresponding driving decision information based on the screen shot image and the driving decision model in real time, and inputs the driving decision information into the vehicle control model to obtain the corresponding robot control instructions. The driving control part controls the driving operation mechanism in real time based on the robot control instructions. The simulated driving mechanism performs the simulated driving operation, so that the simulated driving mechanism sends the driving simulation information corresponding to the simulated driving operation to the computer, and further enables the driving simulation screen generation control unit to generate a new driving simulation based on the high simulation racing game engine and the received driving simulation information. The driving scene image and control the display screen to display and update, the driving simulation data acquisition and storage unit is used to obtain at least all the robot control instructions generated by the control instruction generation unit and the corresponding screen shot images as driving simulation data and store them accordingly. The decision model iterates The driving decision-making model is iteratively updated according to all driving simulation data stored in the driving simulation data acquisition and storage section.

The autonomous driving test platform based on the autonomous driving robot provided by the present invention may also have such technical features, wherein the computer further has a driving simulation result generation unit, the central controller also has a model verification unit, and the driving score index generation unit is based on the high The simulation racing game engine generates the corresponding scoring index according to the game results generated by the driving simulation information and the predetermined scoring method. The model verification output unit verifies the driving decision model and the vehicle control model according to the scoring index, and outputs the model for evaluating the quality of the model. Evaluation results.

The autonomous driving test platform based on the autonomous driving robot provided by the present invention may also have such technical features, wherein the driving decision model includes a decision generation module and a decision correction module, and the control instruction generation unit has: a screen image preprocessing unit for Preprocess a pair of screen shot images in real time to form an environment image to be input corresponding to the virtual driving environment, and identify the state parameters of the virtual vehicle generated by the high-simulation racing game engine from the screen shot image as vehicle state data; The preliminary decision generation unit is used to input the environment image to be input into the decision generation module of the driving decision model to generate preliminary decision information; the driving decision correction unit is used to input the preliminary decision information and vehicle state data into the decision correction module to output the driving decision information ; and an instruction generation unit for inputting driving decision information into a vehicle control model to generate robot control instructions.

The automatic driving test platform based on the autonomous driving robot provided by the present invention may also have such technical features, wherein the driving robot can also sit on the driving seat of the real vehicle to be tested and operate the driving mechanism of the vehicle to be tested, driving The robot also has a monocular camera, a high-precision positioning system and a pose sensor. The monocular camera is used to shoot the dashboard of the vehicle to be tested and output the captured image of the instrument in real time to the central controller. The high-precision positioning system is used to The location of the driving robot is located and the corresponding high-precision positioning information is generated in real time. The pose sensor is used to detect the pose and obtain the attitude information of the vehicle to be tested. The binocular camera is also used to shoot the road scene of the vehicle to be tested. and output a pair of real-scene shooting images obtained by binocular shooting in real time to the central controller, and the control instruction generation part also has: a real-scene image preprocessing unit, which is used for real-time shooting of a pair of real-scene images, instrument shooting images, and high-precision positioning information. and attitude information for preprocessing, forming an environment image to be input corresponding to the actual driving environment according to the real scene shooting image, and identifying the instrument information in the instrument shooting image, and further using the instrument information, high-precision positioning information and attitude information as the vehicle to be tested vehicle status data.

The automatic driving test platform based on the autonomous driving robot provided by the present invention may also have such technical features, wherein the simulated driving mechanism at least has a steering wheel, a speed control gear and an accelerator and brake pedal, and the driving operation mechanism at least has a steering wheel Steering manipulators for operating, shifting manipulators for operating speed gears, and mechanical legs for operating accelerator and brake pedals.

The autonomous driving test platform based on the autonomous driving robot provided by the present invention may also have such technical features, wherein the virtual driving environment includes at least one or several effects of dynamic weather, day and night cycle, car body dirt and halo.

Invention action and effect

According to the automatic driving test platform based on the autonomous driving robot of the present invention, since the car driving simulator uses a high-simulation racing game engine to generate a driving scene image that simulates the virtual driving environment and displays it on the display screen, and the driving robot passes the driving decision model and The vehicle control model processes the screen image captured by the camera to form the corresponding robot control command, and further drives the robot to drive the simulated driving mechanism of the car driving simulator according to the command. The driving decision model in the driving robot is iteratively trained and verified, which strengthens the model's perception and decision-making capabilities in different scenarios. Combining both a driving robot and a driving simulator enables a complete set of virtual driving behaviors from the simulator to obtain a high-fidelity simulated environment to the robot driving a car in a virtual environment. Through the automatic driving simulation platform of the present invention, the game engine can be used to carry out a large amount of initial training on the unmanned driving algorithm, which reduces the cost of actual vehicle testing and the construction of professional software, which is helpful for various enterprises to carry out automatic driving of various types of vehicles. The construction of human-driving algorithms greatly saves the time and cost required to build unmanned algorithms.

Further, since the driving robot has a robot body matched with the driver's seat, on which is installed a driving operation actuator composed of a gear shift, steering manipulator, accelerator, and brake mechanical legs, etc. Under the condition of modification, it can be installed in the cab non-destructively, simulating the automatic driving of the vehicle by human drivers in harsh conditions and dangerous environments, or in a virtual environment, which is helpful for relevant enterprises or personnel to use the driving robot to improve the unmanned driving algorithm. verification and testing.

Description of drawings

Fig. 1 is the structural block diagram of the automatic driving test platform based on autonomous driving robot in the embodiment of the present invention;

2 is a schematic structural diagram of an autonomous driving test platform based on an autonomous driving robot in the embodiment of the present invention;

3 is a structural block diagram of a computer in an embodiment of the present invention;

4 is a schematic diagram of a driving scene image in an embodiment of the present invention;

Fig. 5 is the structural block diagram of the central controller of the embodiment of the present invention;

6 is a schematic structural diagram of an end-to-end unmanned driving in an embodiment of the present invention;

7 is a flowchart of a simulated driving process in an embodiment of the present invention;

8 is a flow chart of a real vehicle driving process in an embodiment of the present invention; and

FIG. 9 is a schematic diagram of fusion of driving models in an embodiment of the present invention.

Detailed ways

In order to make the technical means, creative features, goals and effects realized by the present invention easy to understand, the following describes the automatic driving test platform based on the autonomous driving robot of the present invention in conjunction with the embodiments and the accompanying drawings.

<Example 1>

1 is a structural block diagram of an autonomous driving test platform based on an autonomous driving robot in an embodiment of the present invention, and FIG. 2 is a schematic structural diagram of an autonomous driving test platform based on an autonomous driving robot.

As shown in FIG. 1 and FIG. 2 , the autonomous driving test platform 100 based on the autonomous driving robot includes a car driving simulator 101, a driving robot 102 and a communication network 103.

The communication network 103 is a 5G wireless network, and the car driving simulator 101 is connected to the driving robot 102 through the communication network 103 in communication.

The car driving simulator 101 is a car game simulation platform that integrates an ultra-high computer PC, a driving operating mechanism (steering wheel, shift lever, seat, pedals, etc.) and a display screen. In this embodiment, the automobile driving simulator 101 has a simulator frame 11, a driver's seat 12, a display screen 13, a simulated driving mechanism 14, and a computer 15.

The simulator frame 11 is used to fix all the hardware components of the car driving simulator 101, and the simulator frame 11 is a conventional plastic or metal frame.

A driver's seat 12 is provided on the simulator frame 11 for allowing the driving robot 102 to sit on the driver's seat 12 . In this embodiment, the configuration of the driver's seat 12 is the same as that of a general driver's seat 12 .

The display screen 13 is disposed in front of the driver's seat 12 and faces the driver's seat 12 . When the driving robot 102 sits on the driver's seat 12 , it can face the display screen 12 directly.

The simulated driving mechanism 14 is arranged around the driver's seat 12, and has simulated driving components such as a steering wheel, a shift lever, an accelerator and a brake pedal (ie, an accelerator pedal, a brake pedal and a clutch pedal). The layout position of the vehicle (hereinafter referred to as the real vehicle) is consistent.

In this embodiment, when each simulated driving mechanism is operated, corresponding driving simulation information, such as steering information, braking information, acceleration information, etc., will be generated. The driving simulation mechanism 14 outputs the generated driving simulation information to the computer 15 in real time, so that the computer 15 simulates the behavior of the vehicle according to the driving simulation information.

FIG. 3 is a structural block diagram of a computer in an embodiment of the present invention.

As shown in FIG. 3 , the computer 15 includes a simulation engine storage unit 151, a driving simulation screen generation control unit 152, a driving score index generation unit 153, a simulator communication unit 154, and a simulator control unit 155 for controlling the above-mentioned units,

The simulation engine storage unit 151 stores a plurality of different high-simulation racing game engines.

In this embodiment, the high-simulation racing game engine is a variety of vehicle game engines with high simulation effects, such as world-class racing game engines such as World Rally Championship 5, Need for Speed 20, Forza Motorsport: Horizon 4, Game Dust 4, etc. . Through these game engines, the computer 15 can build a highly realistic virtual driving environment, and can also provide support for dynamic weather systems, day and night cycles, real physical destruction effects, car body dirt, and halo lens displays in the virtual driving environment. effect, so that the shooting environment is more realistic and has sufficient complexity.

The driving simulation screen generation control unit 152 generates a driving scene image capable of simulating a virtual driving environment based on the high-simulation racing game engine stored in the simulation engine storage unit 151, and controls the display screen 12 to display the driving scene image in real time accordingly.

In this embodiment, the driving scene image is a game screen generated based on a high-simulation racing game engine, and the game screen displays at least a virtual environment (such as a track, etc.) and a virtual vehicle (as shown in Figure 4). When the simulator communication unit 154 receives the driving simulation information output when the simulated driving mechanism 14 is operated, the driving scene image will also be updated accordingly, that is, the virtual vehicle in the game screen will perform the corresponding driving behavior according to the driving simulation information ( Such as acceleration, braking and other behaviors), and the virtual environment will also change accordingly.

The driving score index generation unit 153 generates a corresponding score index based on the game result generated by the high simulation racing game engine according to the driving simulation information and a predetermined scoring method.

In this embodiment, the scoring method is to calculate the length of time for the virtual vehicle to move a fixed number of laps, and use the time as a scoring index. When the time in the scoring index is shorter, it means that the virtual vehicle drives more smoothly in the virtual environment (i.e., the fewer accidents occur), and through the scoring index, it is possible to judge whether the driving robot 102 is unmanned or not.

In other solutions of the present invention, the scoring method can also judge whether the driving robot can operate the virtual vehicle to complete overtaking and obstacle avoidance actions, and provide corresponding scoring indicators, thereby helping the unmanned testers to better evaluate the performance according to these scoring indicators. The advantages and disadvantages of the control algorithm of the driving robot 102 and whether it can be practically applied can be judged appropriately.

The driving robot 102 includes a robot body 21, a binocular camera 22, a monocular camera 23, a high-precision positioning system 24, a pose sensor 25, a 5G communication module 26, a driving operating mechanism 27, and a central controller 28.

The robot body 21 is used to mount and fix all the hardware components of the driving robot 102. In this embodiment, the robot body 21 is a humanoid body, which can imitate a driver to sit on the driver's seat 12.

The binocular camera 22 is used for real-time monitoring on the display screen 13 when the driving robot 102 is installed on the car driving simulator 101 (ie, the driving robot 102 is seated on the driver's seat 12 to perform an unmanned test, hereinafter referred to as a virtual test). The displayed driving scene image is captured and the captured screen captured image is output to the central controller 28 in real time.

In addition, in order to ensure the shooting effect of the binocular camera 22, such as avoiding noise such as streaks in the screen shot image, the configurations of the car driving simulator 101 and the driving robot 102 need to be adjusted accordingly before starting the car driving simulator 101 and the driving robot 102.

The monocular camera 23 is used for when the driving robot 102 is set on the real vehicle (that is, the driving robot 102 is allowed to sit on the actual vehicle to perform the unmanned test, hereinafter referred to as the real vehicle test), to monitor the vehicle displayed on the instrument of the vehicle. The data is captured to obtain the meter captured image.

In this embodiment, the monocular camera 23 is arranged on the head of the driving robot through the pan-tilt mechanism, and the pan-tilt mechanism can automatically adjust the position and angle of the lens to make the center of the lens of the monocular camera 23 face the center of the instrument of the reagent vehicle.

In this embodiment, since the monocular camera 23 cannot obtain position information during real-time shooting, the binocular camera 22 will also shoot the driving road scene in front of the vehicle and output the binocularly captured images during the real-vehicle test. A pair of real-scene shooting images enables the central controller 28 to directly measure the distance of the scene ahead (the range captured by the images) by calculating the parallax of the two images, without judging what type of obstacles appear ahead.

The high-precision positioning system 24 is an additive hybrid of the GPS/BDS satellite navigation and positioning system and its enhanced system RTK, which can perform high-precision positioning on the position of the driving robot 102 and output the corresponding information when the driving robot 102 performs a real vehicle test. High-precision positioning information.

The pose sensor 25 is provided on the robot body 21, and is used to acquire the attitude information of the vehicle itself.

In this embodiment, during the simulation test, the monocular camera 23 , the high-precision positioning system 24 and the pose sensor 25 may not work. During the unmanned driving test of the real vehicle, the above three will be collected in real time together with the binocular camera 22, and at this time, the binocular camera 22 will shoot the actual road scene in front of the vehicle and obtain the corresponding real scene shooting. image.

The 5G communication module 26 is provided on the robot body 21, and is used for the communication connection between the central controller 28 and the computer 15.

The driving operation mechanism 27 is used for driving the simulated driving mechanism 14 of the automobile driving simulator 101 .

In this embodiment, the driving operating mechanism 27 includes a manipulator and a mechanical leg corresponding to each simulated driving component in the simulated driving mechanism 14. Specifically: the driving operating mechanism 27 includes a steering manipulator 261 for operating the steering wheel, a The shifting manipulator 262 for operating the speed control gear and the mechanical leg 263 for operating the accelerator and brake pedals (for clutching, braking and stepping on the accelerator). Since this type of manipulator (leg) and the corresponding control method are in the prior art, they will not be repeated here.

The central controller 28 is used to control the driving operation mechanism 27 to perform corresponding driving operations on the simulated driving mechanism 14.

FIG. 5 is a structural block diagram of a central controller according to an embodiment of the present invention.

As shown in FIG. 5 , the central controller 28 includes a driving model storage unit 281 , a control command generation unit 282 , a driving control unit 283 , a driving simulation data acquisition and storage unit 284 , a decision model iteration unit 285 , a model verification unit 286 , and a controller. The communication unit 287 and the controller control unit 288 for controlling the above-mentioned units.

The driving model storage unit 281 stores a driving decision model for generating driving decision information capable of deciding the driving operation of the vehicle, and a vehicle control model for generating corresponding robot control commands based on the driving decision.

Among them, the driving decision model is a convolutional neural network, which can make decisions on the driving behavior of the vehicle (ie, output driving decision information) according to the images captured by the driving robot 102, such as outputting the need to step on the brake at a red light to stop, and when driving How much driving speed to maintain, whether to steer and other driving behaviors. The driving decision model needs a lot of iterative training before it can output the correct driving behavior.

The vehicle control model is a conventional driving robot control algorithm. The algorithm can generate specific robot control instructions according to the driving decision information output by the driving decision model. For example, when the driving decision information is braking, the algorithm will generate the corresponding control for pressing the brake pedal. instruction.

In this embodiment, the driving decision model is composed of a decision generation module and a decision correction module.

The control instruction generation unit 282 has a screen image preprocessing unit 2821, a real image preprocessing unit 2822, a decision generation unit 2823, and an instruction generation unit 2825.

The screen image preprocessing unit 2821 is used to preprocess a pair of screen shot images output by the binocular camera 22 in real time during the virtual test, to form an environment image to be input corresponding to the virtual driving environment, and to identify from the screen shot images. The state parameters of the virtual vehicle generated by the high-simulation racing game engine are obtained as vehicle state data.

In this embodiment, since the driving simulation picture displayed on the display screen 13 is simulated and generated by a high-simulation racing game engine, the picture displays the instrument image of the virtual vehicle (such as a speedometer), a minimap (representing the location of the vehicle), etc. The state parameters of the virtual vehicle, the screen image preprocessing unit 2821 can identify and identify these state parameters through the screen shot image.

The real scene image preprocessing unit 2822 is used to real-timely capture a pair of real scene shooting images output by the binocular camera 22, the instrument shooting images output by the monocular camera 23, the high-precision positioning information output by the high-precision positioning system 24, and the high-precision positioning information output by the high-precision positioning system 24. The attitude information output by the attitude sensor 25 is preprocessed, and an environment image to be input corresponding to the actual driving environment is formed according to the real shot image, and the meter information in the meter shot image is recognized, and the meter information, high-precision positioning information and attitude information are further identified. As the vehicle status data of the vehicle to be tested.

The preliminary decision generation unit 2823 is used to input the environment image to be input into the decision generation module of the driving decision model to generate preliminary decision information.

The driving decision correction unit 2824 is configured to input the preliminary decision information and vehicle state data output by the preliminary decision generation unit 2823 into the decision correction module to output driving decision information.

The instruction generation unit 2825 is used to input the driving decision information into the vehicle control model to generate robot control instructions.

In this embodiment, the central controller 28 mainly uses machine vision to realize unmanned driving, and the principle is an end-to-end unmanned driving technology based on a deep neural network.

FIG. 6 is a schematic structural diagram of an end-to-end unmanned driving in an embodiment of the present invention.

As shown in Figure 6, the core of end-to-end autonomous driving is a deep learning model. Through real-time collection of image data of different road conditions during driving, the driver's control parameters of the car under different road conditions are recorded at the same time. These data are input to the deep learning model as training data for training. When using the deep learning model to control the automatic driving of the car, the binocular camera is used to collect real-time road conditions images (that is, real-life images), and the depth information is measured, and the images are fused and input into the deep learning model to obtain the car wire control parameters (that is, the robot). control instructions), so that the robot can be controlled to manipulate the car to drive automatically, while the depth information enters the next layer of decision-making network.

The driving control unit 283 is used to control the driving operation mechanism 27 in real time according to the robot control command generated by the control command generation unit 282.

In this embodiment, the driving simulation image generation control unit 152 generates a driving scene image in real time based on the high-simulation racing game engine and the driving simulation information output by the simulated driving mechanism 14 and controls the display screen 12 to display accordingly, and the control instruction generation unit 282 The monocular camera 23 generates a robot control command from the screen image captured by the display screen 12, and the driving control unit 283 controls the driving operation mechanism 27 to perform driving operations on the simulated driving mechanism 14 according to the robot control command, thereby forming a possible The cyclic automatic driving process achieves the purpose of controlling the virtual car in the virtual environment, and finally realizes the closed-loop control and algorithm verification of the entire driving behavior.

The driving simulation data acquisition and storage unit 284 is used to acquire all the robot control instructions generated by the control instruction generation unit 282, the corresponding screen shot images, and the score indexes generated by the driving score index generation unit 153 as driving simulation data and store them accordingly.

In addition, in this embodiment, when the driving robot 102 performs an unmanned test of a real vehicle, the driving simulation data acquisition and storage unit 284 also acquires the high-precision positioning information collected by the high-precision localizer 24 and the images captured by the binocular camera 22 at the same time. images as driving simulation data.

The decision model iteration unit 285 iteratively updates the driving decision model according to all the driving simulation data stored in the driving simulation data acquisition and storage unit 284.

The model verification output unit 286 verifies the driving decision model and the vehicle control model according to the scoring index, and outputs a model evaluation result for evaluating the quality of the models.

In this embodiment, the model verification output unit 286 can be output to the terminal held by the tester, so that the tester can judge whether the driving decision model and the vehicle control model can be put into practical use or need to be adjusted according to the result.

The controller communication unit 287 is used for data exchange between the central controller 28 and the computer 15, the monocular camera 23, the binocular camera 22, the high-precision positioning system 24, and the driving operation mechanism 27.

FIG. 6 is a flowchart of a simulated driving process in an embodiment of the present invention.

As shown in FIG. 6, after starting the car driving simulator 101 and the driving robot 102, the following steps are started:

In step S1-1, the driving simulation screen generation control unit 152 in the computer 15 acquires a high-simulation racing game engine from the simulation engine storage unit 151, and generates a driving scene image in real time based on the high-simulation racing game engine and controls the display screen 13 to perform the driving. display, and then enter step S1-2;

Step S1-2, the binocular camera 23 shoots the display screen 13 to obtain a screen shot image corresponding to the driving simulation picture, and then enters step S1-3;

Step S1-3, the control instruction generation part 282 generates the corresponding robot control instruction based on the screen shot image captured in step S1-2 and the driving decision model and the vehicle control model stored in the driving model storage part 171, and then enters step S1-4;

Step S1-4, the driving control unit 283 controls the driving operation mechanism 27 to perform the driving operation on the simulated driving mechanism 14 according to the robot control instruction generated in the step S1-3, and then proceeds to the step S1-5;

Step S1-5, the simulated driving mechanism 14 generates corresponding driving simulation information according to the driving operation of step S1-4 and sends it to the computer 15, and then enters step S1-6;

In step S1-6, the driving simulation screen generation control unit 152 generates a new driving scene image based on the high-simulation racing game engine obtained in step S1-1 and the driving simulation information sent in step S1-5 and controls the display screen 13 to display, and then Enter step S1-7;

Step S1-7, the central controller 28 judges whether to end one round of simulated driving according to the preset simulated driving end condition, if otherwise, it goes to step S1-2, and if so, it goes to step S1-8;

Step S1-8, the driving simulation data acquisition and storage unit 284 obtains all robot control instructions and corresponding screen shot images generated by a round of simulated driving as driving simulation data and stores correspondingly, and then enters step S1-9;

Step S1-9, the decision model iteration unit 285 completes one round of iteration on the driving decision model stored in the driving model storage unit 171 according to the driving simulation data obtained in the driving simulation data storage unit 284, and then proceeds to step S1-10;

In step S1-10, the central controller 28 judges whether the driving decision-making model has reached the preset iterative completion condition, if not, it goes to step S1-1, and if so, it goes to the end state.

Through the above simulation driving process, the driving decision-making model can be preliminarily improved, and the driving decision-making model can be more accurately determined based on the images captured by the driving robot. At this time, the driving robot has a preliminary unmanned driving. ability.

In this embodiment, in steps S1-1 to S1-7 of the above-mentioned simulated driving process, the driving score index generation unit 153 will generate a corresponding score index, and the end condition of the simulated driving is to judge whether to end a round according to whether the score index is generated or not. Simulate driving. In other solutions of the present invention, the simulated driving end condition can also be set according to actual needs, for example, whether to end a round of simulated driving is determined according to whether the driving duration reaches a preset threshold.

In this embodiment, the iteration completion condition is whether the number of rounds of iteration reaches a preset threshold. In other solutions of the present invention, the iteration completion condition can also be set according to actual needs, for example, it is to detect whether the score index generated by the driving score index generation unit 153 reaches a predetermined standard or the like.

After the iteration of the driving decision model is completed through the above-mentioned simulated driving process, and the unmanned capability of the driving robot 102 is tested and passed through the above-mentioned process and the scoring index again, the driving robot 102 can also be placed in the driver's seat of the actual vehicle. Carry out a real vehicle test.

FIG. 7 is a flowchart of a real vehicle driving process of a driving robot in an embodiment of the present invention.

As shown in FIG. 7, after placing the driving robot 102 on the driver's seat of the actual vehicle and starting the driving robot 102, the following steps are started:

In step S2-1, the monocular camera 23 captures the instrument panel of the actual vehicle and outputs the captured instrument image to the central controller 28, and then proceeds to step S2-2;

Step S2-2, the binocular camera 22 shoots the road scene in front of the actual vehicle and outputs a pair of real-scene shooting images obtained by binocular shooting respectively to the central controller 28, and then enters step S2-3;

Step S2-3, the high-precision positioning system 24 locates the position of the driving robot 102 and outputs the corresponding high-precision positioning information to the central controller 28, and then enters step S2-4;

In step S2-4, the control instruction generation part preprocesses the meter photographed image outputted in step S2-1, the real scene photographed image outputted in step S2-2, and the high-precision positioning information outputted in step S2-3, and generates the data to be input, and generates the data to be input. Based on the data to be input and the driving decision model and vehicle control model stored in the driving model storage unit 171, the corresponding robot control command is generated, and then the process goes to step S2-5;

In step S2-5, the driving control unit 283 controls the driving operation mechanism 27 to perform the driving operation on the simulated driving mechanism 14 according to the robot control instruction generated in the step S2-4, and then enters the step S2-5.

Step S2-6, the central controller 28 judges whether to end one round of real car driving according to the preset real vehicle driving end condition, if otherwise, enter step S2-1, if so, enter step S2-7;

In step S2-7, the driving simulation data acquisition and storage unit 284 acquires all the robot control instructions generated by a round of real vehicle driving, as well as the corresponding instrument shooting images, real scene shooting images and high-precision positioning information as driving simulation data and correspondingly stored, and then enter the Step S2-8;

Step S2-8, the decision model iteration unit 285 completes one round of iterations on the driving decision model stored in the driving model storage unit 171 according to the driving simulation data obtained in the driving simulation data storage unit 284, and then proceeds to step S2-9;

In step S2-9, the central controller 28 judges whether the driving decision-making model has reached the preset iterative completion condition, if not, it goes to step S2-1, and if so, it goes to the end state.

Through the above process, the driving robot trained based on the automatic driving simulation platform of the present invention can be used for real vehicle testing and verification.

In addition, the final driving decision model obtained after simulating the driving process and the actual vehicle driving process can be fused with the vehicle control model, as shown in Figure 8, combined with the vehicle model of the actual vehicle, a vehicle dynamics model-based model can be formed. The vehicle control mathematical model, which can be integrated into the system of the driverless car, combined with all kinds of data collected by the sensors of the driverless car for driverless control.

In addition, the central controller 28 can also have a data recording and output interface, which can export the entire driving behavior data (driving behaviors such as acceleration, braking, and shifting) to a third-party software platform such as MATLAB for further analysis (different drivers). There are different driving habits, and the driving behavior can be further optimized through big data analysis to achieve the purpose of smoothness and economy of vehicle operation).

Example function and effect

According to the autonomous driving test platform based on the autonomous driving robot provided in this embodiment, since the car driving simulator uses a high-simulation racing game engine to generate a driving scene image that simulates a virtual driving environment and displays it on the display screen, while the driving robot decides by driving The model and the vehicle control model process the screen image captured by the monocular camera to form the corresponding robot control command, and further drive the robot to drive the simulated driving mechanism of the car driving simulator according to the command. Therefore, through this simulation The driving decision model in the driving robot is iteratively trained and verified in the form of simulation, which strengthens the perception and decision-making ability of the model in different scenarios. Combining both the driving robot and the driving simulator enables a complete set of virtual driving behaviors from the simulator obtaining a high-fidelity simulated environment to the robot driving a car in a virtual environment. Through the automatic driving simulation platform of the present invention, the game engine can be used to carry out a large amount of initial training on the unmanned driving algorithm, which reduces the cost of actual vehicle testing and the construction of professional software, which is helpful for various enterprises to carry out automatic driving of various types of vehicles. The construction of the human driving algorithm greatly saves the time and cost required to construct the unmanned driving algorithm.

The present invention combines the game simulation engine (the game engine authorization fee is only a few hundred yuan) and the semi-physical virtual simulation platform built by the driving robot. In addition to the restoration of a high-fidelity test environment that can meet various application scenarios, the manufacturing cost of the entire system does not exceed One hundred thousand yuan.

Further, since the driving robot has a robot body matched with the driver's seat, on which is installed a driving operation actuator composed of a gear shift, steering manipulator, accelerator, and brake mechanical legs, etc. Under the condition of modification, it can be installed in the cab non-destructively, simulating the automatic driving of the vehicle by human drivers in harsh conditions and dangerous environments, or in virtual environments, which is conducive to the verification and testing of unmanned algorithms by using the driving robot.

At present, unmanned vehicles are not really on the market, and most of them are transformed from traditional vehicles (the transformation cost is high and time-consuming and laborious). Autonomous driving robots can be installed on all kinds of test vehicles without any need for modification of the car, which greatly saves time and energy. Therefore, as a low-cost and cost-effective simulation tool for unmanned training and testing models, it actually solves the problems of the test environment and the test vehicle at the same time, and can be flexibly matched to expand its versatility, such as changing different game engines To replace the test scene, it can greatly reduce the material and time cost of driverless vehicle testing, and has great market application value.

Furthermore, the driving decision-making model obtained through the iterative training of the driving simulation platform of the present invention can also be extracted and integrated with the driving control model to form an unmanned module, which can be set in the system of the unmanned vehicle and installed. Realize unmanned driving.

The above embodiments are only used to illustrate specific embodiments of the present invention, and the present invention is not limited to the description scope of the above embodiments.

For example, in the above-mentioned embodiment, the simulated driving mechanism only includes mechanisms such as a steering wheel, a shift lever, an accelerator and a brake pedal, and the like. In other solutions of the present invention, the simulated driving mechanism may also include a handbrake, an ignition switch, etc., and the driving robot is configured with a corresponding operating arm and operating algorithm, so as to better simulate unmanned driving.

For example, in the above embodiment, the driving model storage unit only stores one driving decision model, and the optimization is completed through multiple iterations. In other solutions of the present invention, the driving model storage unit may also store a variety of driving decision models (for example, corresponding to different driving styles), and these driving decision models are optimized through the automatic driving test platform of the present invention, so as to complete the test Then, let the testers select the most suitable driving decision model according to the test results and use it in practical applications.

Claims (6)

1. an automatic driving test platform based on an autonomous driving robot, is characterized in that, comprises: An automobile driving simulator has a driver's seat, a display screen and a simulated driving mechanism arranged in front of the driver's seat, and a computer connected in communication with the display screen and the simulated driving mechanism, respectively; and A driving robot, sitting on the driving seat and operating the simulated driving mechanism, has a robot body matched with the driving seat, a binocular camera and a driving operation mechanism arranged on the robot body, and a robot body matched with the driving seat. The binocular camera, the driving operating mechanism and the computer are respectively connected to the central controller in communication, Wherein, the computer has a simulation engine storage unit and a driving simulation screen generation control unit, The central controller has a driving model storage part, a control instruction generation part, a driving control part, a driving simulation data acquisition and storage part, a decision model iteration part and a controller communication part, The simulation engine storage unit stores a variety of high-simulation racing game engines, The driving model storage unit stores a driving decision model for generating driving decision information capable of making decisions on vehicle driving operations and a vehicle control model for generating corresponding robot control instructions according to the driving decision, The driving simulation screen generation control unit generates a driving scene image simulating a virtual driving environment based on the high-simulation racing game engine and controls the display screen to display accordingly, The binocular camera captures the driving scene image displayed on the display screen in real time, and outputs to the central controller in real time a pair of screen capture images obtained by binocular shooting respectively, Once the controller communication part receives the screen shot image, the control instruction generation part generates the corresponding driving decision information based on the screen shot image and the driving decision model in real time, and inputs the driving decision information The vehicle control model obtains corresponding robot control instructions, The driving control unit controls the driving operation mechanism to perform a simulated driving operation on the simulated driving mechanism in real time based on the robot control instruction, so that the simulated driving mechanism sends driving simulation information corresponding to the simulated driving operation to the simulated driving mechanism. a computer, further enabling the driving simulation screen generation control unit to generate a new driving scene image based on the high-simulation racing game engine and the received driving simulation information and control the display screen to display and update, The driving simulation data acquisition and storage unit is configured to at least acquire all the robot control instructions generated by the control instruction generation unit and the corresponding captured screen images as driving simulation data and store them accordingly, The decision model iteration unit iteratively updates the driving decision model according to all the driving simulation data stored in the driving simulation data acquisition and storage unit. 2. the automatic driving test platform based on autonomous driving robot according to claim 1, is characterized in that: Wherein, the computer also has a driving simulation result generation unit, The central controller also has a model verification unit, The driving scoring index generating unit generates a corresponding scoring index based on the game result generated by the high-simulation racing game engine according to the driving simulation information and a predetermined scoring method, The model verification output unit verifies the driving decision model and the vehicle control model according to the scoring index, and outputs a model evaluation result for evaluating the quality of the models. 3. the self-driving test platform based on autonomous driving robot according to claim 1, is characterized in that: Wherein, the driving decision model includes a decision generation module and a decision correction module, The control command generation part has: The screen image preprocessing unit is configured to preprocess the pair of screen shot images in real time, form an environment image to be input corresponding to the virtual driving environment, and identify from the screen shot images The state parameters of the virtual vehicle generated by the simulation racing game engine are input as vehicle state data; a preliminary decision generating unit, configured to input the environment image to be input into the decision generating module of the driving decision model to generate preliminary decision information; a driving decision correction unit for inputting the preliminary decision information and the vehicle state data into the decision correction module to output the driving decision information; and An instruction generation unit, configured to input the driving decision information into the vehicle control model to generate the robot control instruction. 4. the automatic driving test platform based on autonomous driving robot according to claim 3, is characterized in that: Wherein, the driving robot can also sit on the driver's seat of the real vehicle to be tested and operate the driving mechanism of the vehicle to be tested, The driving robot also has a monocular camera, a high-precision positioning system and a pose sensor, The monocular camera is used to photograph the instrument panel of the vehicle to be tested and output the photographed instrument image to the central controller in real time, The high-precision positioning system is used to locate the position of the driving robot and generate corresponding high-precision positioning information in real time, The pose sensor is used to perform pose detection and obtain the attitude information of the vehicle to be tested, The binocular camera is also used to shoot the road scene where the vehicle to be tested is traveling and output to the central controller in real time a pair of real-world shooting images obtained by binocular shooting respectively, The control instruction generation part also has: A real-scene image preprocessing unit, configured to preprocess the pair of real-scene shooting images, the instrument shooting images, the high-precision positioning information, and the attitude information in real time, and form and actual driving environment according to the real-scene shooting images The corresponding environment image to be input is identified, and the instrument information in the photographed image of the instrument is identified, and the instrument information, the high-precision positioning information and the attitude information are further used as the vehicle state data of the vehicle to be tested. 5. the automatic driving test platform based on autonomous driving robot according to claim 1, is characterized in that: Wherein, the simulated driving mechanism has at least a steering wheel, a speed control gear and an accelerator and brake pedal, The driving operation mechanism has at least a steering manipulator for operating the steering wheel, a shifting manipulator for operating the speed control gear, and a mechanical leg for operating the accelerator and brake pedals. 6. the self-driving test platform based on autonomous driving robot according to claim 1, is characterized in that: Wherein, the virtual driving environment includes at least one or several effects of dynamic weather, day and night cycle, car body dirt and halo.

Images