WO2024131679A1

WO2024131679A1 - Vehicle-road-cloud fused road environment scene simulation method, electronic device, and medium

Info

Publication number: WO2024131679A1
Application number: PCT/CN2023/139202
Authority: WO
Inventors: 王泽兴; 黄荣军; 黄思德; 宛家国; 吴宁; 邹广才; 原诚寅
Original assignee: 北京国家新能源汽车技术创新中心有限公司
Priority date: 2022-12-20
Filing date: 2023-12-15
Publication date: 2024-06-27
Also published as: CN116244902A

Abstract

Disclosed in the present application are a vehicle-road-cloud fused road environment scene simulation method, an electronic device, and a medium. The method comprises: acquiring and transmitting vehicle-end information by means of vehicle-end sensors; acquiring and transmitting road-end information by means of road-end sensors; acquiring and transmitting position information of all road traffic participants by means of a cloud-end communication tool; loading the vehicle-end information, the road-end information and the position information of all the road traffic participants; and performing sensor information fusion on said information, so as to perform real-time simulation scene modeling by means of a real-time simulation modeling tool. In a laboratory environment, the present invention can implement vehicle-road-cloud collaborative computing and mutual fusion tests in loaded real road environment scenes, not only solves the problem of insufficient scene authenticity in vehicle-road-cloud collaborative tests in environment simulation test modes, but also solves the problems of unsafe vehicle-road-cloud collaborative mutual fusion tests and unrepeatable test scenes in real vehicle road test environments.

Description

Road environment scene simulation method, electronic equipment and medium for vehicle-road-cloud fusion

Technical Field

The present invention relates to the field of road environment scene simulation, and more specifically, to a vehicle-road-cloud fusion road environment scene simulation method, electronic equipment and medium.

Background technique

In the new round of scientific and technological revolution and industrial transformation, cars are no longer just a means of transportation, but have gradually transformed into comprehensive technological products that integrate intelligent interaction, automatic control, external communication, artificial intelligence, etc. After the deep integration of the automotive industry with the Internet of Things, communications and other fields, people can achieve real-time interaction between smart cars and smart roads, and carry out active vehicle control and road collaborative management based on the collection and integration of dynamic traffic information in all time and space, fully realize the effective coordination of car-road-cloud, and ultimately achieve the goal of improving traffic efficiency and ensuring traffic safety.

However, if effective collaboration between vehicle, road and cloud is to be achieved, it is necessary to design and test collaborative solutions. Among the existing testing methods, software-in-the-loop testing can solve the problem of testing efficiency, but the authenticity of its test scenarios is insufficient. The test environment provided by closed-field testing and road testing is real, but its safety is insufficient in the early stages of research and development. Especially for road testing, not only can it not meet the requirements of safe driving in the early stages of research and development, but the road scenes cannot be reproduced, making it impossible to carry out scenario repeatability testing and verification of intelligent driving electronic control systems.

Therefore, it is necessary to develop a road environment scene simulation method, electronic equipment and medium for vehicle-road-cloud fusion.

The information disclosed in the background technology section of the present invention is only intended to deepen the understanding of the general background technology of the present invention, and should not be regarded as acknowledging or suggesting in any form that the information constitutes the prior art already known to those skilled in the art.

Summary of the invention

The present invention proposes a road environment scene simulation method, electronic device and medium for vehicle-road-cloud integration, which can realize vehicle, road and cloud collaborative computing and integration testing under real road environment scene loading in a laboratory environment, which not only solves the problem of insufficient scene authenticity in vehicle, road and cloud collaborative testing under a ring simulation test mode, but also solves the problems of unsafe vehicle, road and cloud collaborative integration testing and non-repeatable test scenes under a real vehicle road test environment.

In a first aspect, an embodiment of the present disclosure provides a method for simulating a road environment scene with vehicle-road-cloud fusion, including:

Acquire and transmit vehicle-side information through vehicle-side sensors;

Acquire and transmit roadside information through roadside sensors;

Acquire and transmit the location information of all road traffic participants through cloud communication tools;

Loading the vehicle-side information, the road-side information and the location information of all road traffic participants;

The vehicle-side information, the road-side information and the location information of all road traffic participants are subjected to sensor information fusion, and then real-time simulation scene modeling is performed through a real-time simulation modeling tool.

Preferably, transmitting the vehicle-side information includes:

Transmit the vehicle-side information to the laboratory vehicle-side receiver in real time through 5G communication technology; or

The vehicle-side information is stored on the vehicle-mounted terminal, and when the required scene is obtained, it is intercepted, and the intercepted vehicle-side information is transmitted to the laboratory vehicle-side receiver.

Preferably, transmitting the vehicle-side information includes:

After the vehicle-side information is acquired, it enters the vehicle-mounted computing unit for processing, acquires the target data of the lightweight road participating elements and transmits it to the laboratory vehicle-side receiver.

Preferably, transmitting the road end information includes:

Transmit the roadside information to the laboratory roadside receiver in real time through 5G communication technology; or

The road-side information is stored on the vehicle-mounted terminal, and is intercepted when the required scene is obtained. The intercepted road-end information is transmitted to the laboratory road-end receiver.

Preferably, transmitting the road end information includes:

After the road end information is acquired, it enters the road load calculation unit for processing, acquires the target data of the lightweight road participating elements and transmits it to the laboratory road end receiver.

Preferably, transmitting the location information of all road traffic participants includes:

Transmit the location information of all road traffic participants in real time to the laboratory cloud receiver via 5G communication technology; or

The vehicle and road ends work together to capture the required scenes, and then transmit the captured location information of all road traffic participants to the laboratory cloud receiver.

Preferably, loading the vehicle-side information, the road-side information and the location information of all road traffic participants includes:

The laboratory server loads the vehicle-side information, the road-side information and the location information of all road traffic participants obtained from the vehicle-side, road-side and cloud to the on-board computing unit, the roadside edge computing unit and the cloud computing unit in the laboratory environment respectively.

Preferably, it also includes:

After the modeling is completed, the real-time simulation scenario is loaded into the laboratory vehicle computing unit through the HIL cabinet to test the scene perception, planning, decision-making and control algorithms. The test results are then transmitted to the vehicle dynamics model or combined with the vehicle simulation motion bench through the HIL cabinet for response and functional evaluation.

As a specific implementation method of the embodiment of the present disclosure,

In a second aspect, an embodiment of the present disclosure further provides an electronic device, the electronic device comprising:

A memory storing executable instructions;

A processor runs the executable instructions in the memory to implement the road environment scene simulation method for vehicle-road-cloud fusion.

In a third aspect, the present disclosure also provides a computer-readable storage medium storing a computer program, which implements the The road environment scene simulation method of vehicle-road-cloud fusion is described.

Its beneficial effects are:

1. Real road environment scenes are transmitted to the laboratory environment in real time or offline via three routes: vehicle-road-cloud. The transmission methods include both large data transmission of original sensor signals and lightweight data transmission of target information of road participating elements.

2. Load the real road environment scene into the vehicle-road-cloud electronic control system to be tested in the laboratory environment, perform integrated collaborative calculation and processing, and output the integrated and processed results to the vehicle dynamics model through the vehicle-side electronic control system to be tested, or combine with the vehicle simulation motion test bench for response and function evaluation;

3. The real road environment scene is transmitted to the laboratory environment through the three routes of vehicle, road and cloud, and then the simulation scene fusion modeling is performed in the real-time simulation modeling tool, and the fusion modeling scene is loaded into the vehicle-side electronic control system to be tested for testing and evaluation of the autonomous driving function.

The methods and apparatus of the present invention have other features and advantages that will be apparent from, or will be described in detail in, the accompanying drawings and subsequent detailed descriptions incorporated herein, which together serve to explain the specific principles of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other objects, features and advantages of the present invention will become more apparent through a more detailed description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, wherein like reference numerals generally represent like components throughout the exemplary embodiments of the present invention.

FIG1 shows a flow chart showing the steps of a method for simulating a road environment scene using vehicle-road-cloud fusion according to an embodiment of the present invention.

FIG2 shows a schematic diagram of a road environment scene simulation system for vehicle-road-cloud fusion according to an embodiment of the present invention.

Detailed ways

The preferred embodiments of the present invention will be described in more detail below. Although the preferred embodiments of the present invention are described below, it should be understood that the present invention can be implemented in various forms and should not be limited to the embodiments set forth herein.

To facilitate understanding of the solutions and effects of the embodiments of the present invention, three specific application examples are given below. Those skilled in the art should understand that the examples are only for facilitating understanding of the present invention, and any specific details thereof are not intended to limit the present invention in any way.

Example 1

As shown in Figure 1, the road environment scene simulation method of vehicle-road-cloud fusion includes: step 101, acquiring and transmitting vehicle-side information through vehicle-side sensors; step 102, acquiring and transmitting road-side information through road-side sensors; step 103, acquiring and transmitting the location information of all road traffic participants through cloud communication tools; step 104, loading the vehicle-side information, road-side information and the location information of all road traffic participants; step 105, performing sensor information fusion of the vehicle-side information, road-side information and the location information of all road traffic participants, and then performing real-time simulation scene modeling through real-time simulation modeling tools.

In one example, transmitting vehicle-side information includes:

Transmit vehicle-side information to the laboratory vehicle-side receiver in real time through 5G communication technology; or

In one example, transmitting vehicle-side information includes:

After acquiring the vehicle-side information, it enters the on-board computing unit for processing, obtains the target data of the lightweight road participating elements and transmits it to the laboratory vehicle-side receiver.

In one example, the transmission path end information includes:

Transmit roadside information to laboratory roadside receivers in real time via 5G communication technology; or

The road-side information is stored on the vehicle-mounted terminal, and when the required scene is obtained, it is intercepted, and the intercepted road-side information is transmitted to the laboratory road-side receiver.

In one example, the transmission path end information includes:

After acquiring the road-end information, it enters the road load calculation unit for processing, obtains the target data of the lightweight road participating elements and transmits it to the laboratory road-end receiver.

In one example, transmitting the location information of all road traffic participants includes:

Transmit the location information of all road traffic participants to the laboratory cloud receiver in real time via 5G communication technology; or

In one example, loading vehicle-side information, road-side information, and location information of all road traffic participants includes:

The laboratory server will load the vehicle-side information, road-side information and location information of all road traffic participants obtained from the vehicle-side, road-side and cloud to the on-board computing unit, roadside edge computing unit and cloud computing unit in the laboratory environment respectively.

In one example, it also includes:

Specifically, the current testing methods for vehicle-road-cloud collaborative solutions include both pure virtual software-in-the-loop testing methods and field testing methods and testing methods on actual roads. Among them, software-in-the-loop testing uses virtual reality data generation, transmission and interaction technology to simulate the driving of an autonomous vehicle in a real road environment, and uses a probability distribution of dangerous scenarios to enhance the simulation method to perform adaptive acceleration testing. Through software-in-the-loop testing, it can significantly save testing time and cost, provide verification results for virtual testing, and provide a more realistic test for actual road testing. Reference data; the closed-field test is similar to a "propositional test". In a closed area, the smart driving car is tested for basic traffic management facilities detection and response capabilities, dynamic and static targets (motor vehicles, non-motor vehicles, pedestrians, obstacles, etc.) recognition and response capabilities in the front lane, driving compliance capabilities, and comprehensive capabilities. Finally, the road test is to put the vehicle on a public road environment to test the vehicle-road-cloud smart driving capabilities. It is a random working condition test based on real usage conditions.

The vehicle-side route transmission of the real road environment scene is shown in the red route map in Figure 2. The real scene of the road environment is obtained through sensors such as cameras, millimeter-wave radars, ultrasonic radars, lidars, GPS/IMU, etc. arranged on the vehicle side. This scene can be transmitted to the laboratory vehicle-side receiver in real time through communication technologies such as 5G, or it can be stored on the vehicle side and intercepted when the required valuable scene is obtained, and then transmitted to the laboratory vehicle-side receiver.

The transmission route can also be shown as the yellow route map in Figure 2. The sensors arranged on the vehicle side, such as cameras, millimeter-wave radars, ultrasonic radars, lidars, GPS/IMU, etc., obtain the real scene of the road environment and enter the on-board computing unit for processing, so as to obtain the target data of the lightweight road participating elements, and then transmit it to the laboratory vehicle-side receiver.

The road-side route transmission of the real road environment scene, as shown in the red route map in Figure 2, obtains the real scene of the road environment through sensors such as cameras, millimeter-wave radars, and lidars arranged on the road side. This scene can be transmitted to the laboratory road-side receiver in real time through communication technologies such as 5G, or it can be intercepted together with the vehicle side to capture valuable scenes and then transmitted to the laboratory road-side receiver.

The transmission route can also be the yellow route shown in Figure 2. The real scene of the road environment is obtained through sensors such as cameras, millimeter-wave radars, and lidars arranged at the road side, and then enters the roadside edge computing unit for processing, thereby obtaining the target data of the lightweight road participating elements, and then transmitting it to the laboratory road side receiver.

The cloud route transmission of the real road environment scene, the cloud obtains the location information of all road traffic participants through communication tools, and transmits this information to the laboratory cloud in real time through 5G and other communication technologies. It can also intercept valuable scenes together with the vehicle side and the road side, and then transmit them to the laboratory cloud receiver.

The laboratory vehicle-side receiver, road-side receiver and cloud-side receiver receive the real road environment scene. The vehicle-side receiver, road-side receiver and cloud-side receiver in the laboratory environment receive the real road environment scene from the vehicle-side, road-side and cloud-side through 5G real-time communication or periodic copying, including the original sensor signal information (as shown in the red road map in Figure 2) or the target information of the road traffic participating elements after processing by the on-board computing unit and the roadside edge computing unit.

The received information is stored on the one hand, and can be loaded and tested to the in-the-loop test system on the other hand.

When a real road environment scenario is required for load testing, the laboratory server will load the information obtained from the vehicle, road and cloud to the on-board computing unit, roadside edge computing unit and cloud computing unit in the laboratory environment respectively, and transmit and load the real road environment scene to the on-board computing unit, roadside edge computing unit and cloud computing unit in the laboratory environment; the number of on-board computing units and roadside edge computing units in the laboratory environment is uncertain, and needs to be arranged accordingly according to the number of input real road environment vehicles and roadside edge computing units.

The on-board computing unit, roadside edge computing unit and cloud computing unit in the laboratory environment interact and process in real time according to the real road environment scenes loaded by each of them. The real road environment scene data can be divided into target data and raw sensor data.

Regarding the processing of target data, the target-level information data received by the vehicle-side receiver, the road-side receiver and the cloud-side receiver are directly transmitted to the on-board computing unit for data fusion, and real-time simulation scenario modeling is performed through real-time simulation modeling tools.

For the raw sensor data, the on-board computing unit receives the vehicle-side receiver data and completes the vehicle-side real road environment scene target recognition and tracking, the roadside edge computing unit receives the road-side receiver data and completes the road-side real road environment scene target recognition and tracking, and the cloud computing unit receives the vehicle-side receiver data and completes the road-side real road environment scene target recognition and tracking. After receiving the data from the cloud receiver, it is processed. As shown in the green road map in Figure 2, the original sensor information of the real road environment is obtained through the vehicle-side and road-side receivers. This real road environment information can be fused by the high-performance computer unit to form the target information, and then the real-time simulation scene modeling is performed through the real-time simulation modeling tool.

After the modeling is completed, the real-time simulation scene is loaded into the laboratory vehicle computing unit through the HIL cabinet to test the scene perception, planning, decision-making and control algorithms. The test results are then transmitted to the vehicle dynamics model or combined with the vehicle simulation motion bench through the HIL cabinet for response and functional evaluation.

Finally, the responses of simulated vehicles and vehicles in real road environments can also be compared to conduct comparative evaluation and accuracy verification of the vehicle-road-cloud fusion test results in a laboratory environment.

Example 2

The present disclosure provides an electronic device, which includes: a memory storing executable instructions; a processor running the executable instructions in the memory to implement the above-mentioned vehicle-road-cloud fusion road environment scene simulation method.

An electronic device according to an embodiment of the present disclosure includes a memory and a processor.

The memory is used to store non-temporary computer-readable instructions. Specifically, the memory may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory. The volatile memory may, for example, include random access memory (RAM) and/or cache memory (cache), etc. The non-volatile memory may, for example, include read-only memory (ROM), hard disk, flash memory, etc.

The processor may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and/or instruction execution capabilities, and may control other components in the electronic device to perform desired functions. In one embodiment of the present disclosure, the processor is used to run the computer-readable instructions stored in the memory.

Those skilled in the art should understand that in order to solve the technical problem of how to obtain a good user experience, the present embodiment may also include well-known structures such as a communication bus and an interface. Well-known structures should also be included in the protection scope of the present disclosure.

For detailed description of this embodiment, reference may be made to the corresponding descriptions in the aforementioned embodiments, which will not be repeated here.

Example 3

An embodiment of the present disclosure provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the vehicle-road-cloud fusion road environment scene simulation method.

According to the computer-readable storage medium of the embodiment of the present disclosure, non-transitory computer-readable instructions are stored thereon. When the non-transitory computer-readable instructions are executed by a processor, all or part of the steps of the above-mentioned methods of each embodiment of the present disclosure are executed.

The above-mentioned computer-readable storage media include, but are not limited to: optical storage media (e.g., CD-ROM and DVD), magneto-optical storage media (e.g., MO), magnetic storage media (e.g., magnetic tape or mobile hard disk), media with built-in rewritable non-volatile memory (e.g., memory card) and media with built-in ROM (e.g., ROM box).

Those skilled in the art should understand that the purpose of the above description of the embodiments of the present invention is only to exemplarily illustrate the beneficial effects of the embodiments of the present invention, and is not intended to limit the embodiments of the present invention to any given examples.

The embodiments of the present invention have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments.

Claims

A vehicle-road-cloud fusion road environment scene simulation method, characterized by comprising:

Acquire and transmit vehicle-side information through vehicle-side sensors;

Acquire and transmit roadside information through roadside sensors;

Acquire and transmit the location information of all road traffic participants through cloud communication tools;

Loading the vehicle-side information, the road-side information and the location information of all road traffic participants;

The vehicle-side information, the road-side information and the location information of all road traffic participants are subjected to sensor information fusion, and then real-time simulation scene modeling is performed through a real-time simulation modeling tool.
According to the vehicle-road-cloud fusion road environment scene simulation method of claim 1, wherein transmitting the vehicle-side information comprises:

Transmit the vehicle-side information to the laboratory vehicle-side receiver in real time through 5G communication technology; or

The vehicle-side information is stored on the vehicle-mounted terminal, and when the required scene is obtained, it is intercepted, and the intercepted vehicle-side information is transmitted to the laboratory vehicle-side receiver.
According to the vehicle-road-cloud fusion road environment scene simulation method of claim 1, wherein transmitting the vehicle-side information comprises:

After the vehicle-side information is acquired, it enters the vehicle-mounted computing unit for processing, acquires the target data of the lightweight road participating elements and transmits it to the laboratory vehicle-side receiver.
According to the vehicle-road-cloud fusion road environment scene simulation method of claim 1, wherein transmitting the road end information comprises:

Transmit the roadside information to the laboratory roadside receiver in real time through 5G communication technology; or

The road-side information is stored on the vehicle-mounted terminal, and is intercepted when the required scene is obtained. The intercepted road-end information is transmitted to the laboratory road-end receiver.
According to the vehicle-road-cloud fusion road environment scene simulation method of claim 1, wherein transmitting the road end information comprises:

After the road end information is acquired, it enters the road load calculation unit for processing, acquires the target data of the lightweight road participating elements and transmits it to the laboratory road end receiver.
According to the vehicle-road-cloud fusion road environment scene simulation method of claim 1, wherein transmitting the location information of all road traffic participants comprises:

Transmit the location information of all road traffic participants in real time to the laboratory cloud receiver via 5G communication technology; or

The vehicle and road ends work together to capture the required scenes, and then transmit the captured location information of all road traffic participants to the laboratory cloud receiver.
According to the vehicle-road-cloud fusion road environment scene simulation method of claim 1, wherein loading the vehicle-side information, the road-side information and the location information of all road traffic participants comprises:

The laboratory server loads the vehicle-side information, the road-side information and the location information of all road traffic participants obtained from the vehicle-side, road-side and cloud-end to the on-board computing unit, the roadside edge computing unit and the cloud computing unit in the laboratory environment respectively.
The method for simulating a road environment scene by integrating vehicle, road and cloud according to claim 1, further comprising:

After the modeling is completed, the real-time simulation scenario is loaded into the laboratory vehicle computing unit through the HIL cabinet to test the scene perception, planning, decision-making and control algorithms. The test results are then transmitted to the vehicle dynamics model or combined with the vehicle simulation motion bench through the HIL cabinet for response and functional evaluation.
An electronic device, characterized in that the electronic device comprises:

A memory storing executable instructions;

A processor, wherein the processor runs the executable instructions in the memory to implement the road environment scene simulation method for vehicle-road-cloud fusion according to any one of claims 1-8.
A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program, which, when executed by a processor, implements the road environment scene simulation method for vehicle-road-cloud fusion according to any one of claims 1-8.