WO2020199057A1

WO2020199057A1 - Self-piloting simulation system, method and device, and storage medium

Info

Publication number: WO2020199057A1
Application number: PCT/CN2019/080693
Authority: WO
Inventors: 黎晓键
Original assignee: 深圳市大疆创新科技有限公司
Priority date: 2019-03-30
Filing date: 2019-03-30
Publication date: 2020-10-08
Also published as: CN111316324A

Abstract

A self-piloting simulation system, method and device, and a storage medium. The system comprises: a motion module for simulating a simulated travel process, in a simulated travel environment, of a mobile platform; a camera module for determining a camera parameter of a camera of the mobile platform; a rendering module for determining, according to the camera parameter, an emulated environment image obtained by means of the camera photographing the simulated travel environment; and a marking module for marking the emulated environment image, wherein the rendering module is also used for carrying out image rendering on the marked emulated environment image to obtain a marked image. The self-piloting simulation system can emulate a real self-piloting simulation scenario, can simulate a travel process, in a simulated travel environment, of a mobile platform, can simulate an emulated environment image obtained by means of a camera on the mobile platform photographing the simulated travel environment, and can finally carry out freespace detection on the emulated environment image, thereby obtaining a marked image used for assisting the self-piloting simulation system with carrying out route planning.

Description

Automatic driving simulation system, method, equipment and storage medium

Technical field

This application relates to the field of automatic driving technology, and in particular to an automatic driving simulation system, method, device, and storage medium.

Background technique

With the development of autonomous driving technology, self-piloting automobiles (Self-piloting Automobile) have begun to become a research hotspot. Self-driving cars, also known as driverless cars, computer-driven cars, or wheeled mobile robots, are smart cars that realize driverless driving through a computer system. Self-driving cars rely on artificial intelligence, visual computing, radar, monitoring devices, and global positioning systems to work together to allow computers to automatically and safely operate motor vehicles without any human active operation.

Although the market for self-driving cars has great potential, there are few companies that can produce self-driving cars. This is because many self-driving technologies are still in the testing stage. Although self-driving technologies are becoming more advanced and diverse, they are stable Sex cannot be guaranteed. The driving area detection technology is a very important technology in the automatic driving technology, because whether the result of the driving area detection is correct is related to whether the automatic driving can plan the driving route well. How to realize the test and detection of autonomous driving technology at the simulation level has become a hot research issue.

Summary of the invention

The embodiment of the present application provides an automatic driving simulation system, which can simulate a real driving scene that can be used to test automatic driving technology.

In the first aspect, an embodiment of the present application provides an automatic driving simulation system, which includes:

The motion module is used to simulate the simulated driving process of the mobile platform in the simulated driving environment;

The camera module is used to determine the camera parameters of the camera of the mobile platform during the simulation driving process;

A rendering module, configured to determine, according to the camera parameters during the simulated driving process, that the camera of the mobile platform captures the simulated environment image obtained by the simulated driving environment during the simulated driving process;

A marking module, configured to mark the simulated environment image, where at least the drivable area in the simulated environment image is marked when marking;

The rendering module is also used to perform image rendering on the marked simulated environment image to obtain a marked image.

In the second aspect, an embodiment of the present application provides an automatic driving simulation method, and the automatic driving simulation method includes:

Obtain the camera parameters of the mobile platform's camera in the simulation driving process;

Determining, according to the camera parameters in the simulated driving process, the simulated environment image obtained by the camera of the mobile platform during the simulated driving process to capture the simulated driving environment;

Obtain the marked simulation environment image;

Image rendering is performed on the marked simulated environment image to obtain a marked image.

In a third aspect, an embodiment of the present application provides an automatic driving simulation device, the automatic driving simulation device includes a unit for executing the automatic driving simulation method of the second aspect, the automatic driving simulation device includes:

The acquiring unit is used to acquire the camera parameters of the camera of the mobile platform during the simulated driving process;

A rendering unit, configured to determine, according to the camera parameters during the simulated driving process, that the camera of the mobile platform captures the simulated environment image obtained by the simulated driving environment during the simulated driving process;

The acquiring unit is also used to acquire a marked simulation environment image;

The rendering unit is also configured to perform image rendering on the marked simulated environment image to obtain a marked image.

In a fourth aspect, an embodiment of the present application provides an automatic driving simulation device, including a processor and a memory, the processor and the memory are connected to each other, wherein the memory is used to store a computer program, and the computer program includes program instructions , The processor is configured to call the program instructions to execute the method as described in the second aspect

In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer storage medium stores a computer program, the computer program includes program instructions, and the program instructions are executed by a processor. To perform the method described in the second aspect.

The automatic driving simulation system of the present application can simulate a real automatic driving simulation scene. Specifically, the motion module is used to simulate the driving process of the mobile platform in the simulated driving environment, and then the camera module and the rendering module are used to simulate camera shooting The simulation environment image obtained by simulating the driving environment is finally used to detect the driving area of the simulation environment image by the marking module and the rendering module, thereby obtaining the marking image for assisting automatic driving in path planning. It can be seen that this application can simulate a real driving scene and test the driving area detection technology, which provides a reliable algorithm verification for the driving area detection technology. In addition, since the system simulates a real driving scene, the The system can also be used to test more autonomous driving technologies in addition to driving area detection technologies.

Description of the drawings

In order to explain the technical solutions of the embodiments of the present application more clearly, the following will briefly introduce the drawings that need to be used in the description of the embodiments.

FIG. 1 is a schematic block diagram of an automatic driving simulation system provided by an embodiment of the present application;

FIG. 2 is a schematic diagram of a mark image provided by an embodiment of the present application;

FIG. 3 is a schematic flowchart of an automatic driving simulation method provided by an embodiment of the present application;

4 is a schematic flowchart of an automatic driving simulation method provided by another embodiment of the present application;

FIG. 5 is a schematic block diagram of an automatic driving simulation device provided by an embodiment of the present application;

Fig. 6 is a structural block diagram of an automatic driving simulation device provided by an embodiment of the present application.

detailed description

In order to make the purpose, technical solutions and advantages of the present application clearer, the technical solutions of the embodiments of the present application will be described below with reference to the accompanying drawings.

The embodiment of the application provides an automatic driving simulation system, which can simulate a real automatic driving simulation scene, simulate the driving process of a mobile platform in a simulated driving environment, and simulate a camera shooting simulation driving The simulation environment image is obtained from the environment, and finally, the driving area detection is performed on the simulation environment image, so as to obtain a marked image for assisting automatic driving in path planning. Among them, the drivable area is also called Freespace, which is a road on which the mobile platform can travel in automatic driving, and is used to provide path planning for automatic driving to avoid obstacles. The drivable area can be the entire road surface of the road, or a part of the road surface that contains key information of the road (such as road direction information, road midpoint information, etc.). More specifically, the drivable area includes structured pavement, semi-structured pavement, and unstructured pavement. Structured pavement is a pavement with a single pavement structure and road edges, such as urban main roads, high-speed roads, and national roads. , Provincial roads, etc., semi-structured roads are roads with various structures, such as parking lots, squares, etc., and unstructured roads are natural roads without structural layers, such as undeveloped uninhabited areas.

Specifically, as shown in FIG. 1, the above-mentioned automatic driving simulation system includes a motion module 110, a camera module 120, a rendering module 130, and a marking module 140. First, the motion module 110 simulates the simulated driving process of the mobile platform in a simulated driving environment, and The camera module 120 is used to determine the camera parameters of the camera of the mobile platform during the aforementioned simulation driving process of the mobile platform, the camera parameters include at least one of the position information, orientation information, rendering mode, and field of view information of the camera, and then the rendering module 130. According to the camera parameters, determine the simulated environment image obtained by the camera shooting the simulated driving environment, and finally the marking module 140 marks each image element in the simulated environment image, especially the driving area in the simulated environment image needs to be marked, and finally rendered The module 130 renders the marked simulated environment image to highlight each image element in the simulated environment image to obtain a marked image, which is used to assist the mobile platform in path planning during automatic driving to avoid obstacles. Among them, the image elements include images of various objects such as drivable areas, pedestrians, buildings, greenery and roads. The position information in the camera is the position of the camera in the above-mentioned simulated driving environment, and the orientation information is the shooting direction of the camera. The mode is the adjustment method of the image, including resolution change, stretching rotation change, and/or color scale brightness change, etc. The angle of view (fov, angle of view) is the range of angle that the camera can receive the image, also called the field of view. In the imaging range (angle of coverage), the field of view describes the image angle that the camera lens can capture.

It should be noted that the camera parameters of the above-mentioned camera are real data information conforming to the real world, so in the real world, the imaging of the camera can be determined according to the above-mentioned camera parameters, and the mobile platform simulated by the above-mentioned motion module 110 is in a simulated driving environment. The movement process is also in line with the physical rules of the real world, because the aforementioned movement module 110 includes a physics engine that can provide movement information for the interaction between the mobile platform and the simulated driving environment. The physical engine essentially instructs the movement module 110 to simulate the movement process of the mobile platform. The arithmetic rules followed, and the movement process simulated by the physics engine conforms to the physical rules in the real world. Among them, the movement information includes, for example, the spatial position information of the mobile platform, etc. The spatial position information includes the position information and steering information of the mobile platform.

In one implementation, before simulating the above-mentioned driving process, a simulated driving environment containing real-world three-dimensional objects is constructed first. The simulated driving environment includes three-dimensional objects such as terrain, vegetation, weather systems, buildings, and roads. Specifically, the 3D model is constructed according to the size ratio of the 3D object in the real world, and then the 3D model is beautified, so that in addition to the shape, the color and pattern are closer to the object in the real world, and finally an object that can be rotated at will is obtained. And the realistic 3D model displayed at any angle, such as constructing the 3D model of the building with the scale of the building, and then overlaying the color graphics of the building on the 3D model, and lighting and adding shadows to the 3D model to obtain An architectural model that is very similar to a real-world building. By rotating the architectural model, the architectural model can be observed from any perspective.

In an implementation, the aforementioned automatic driving simulation system further includes an output module configured to output the aforementioned mark image. The output mode may be graphic display, network transmission, etc., which is not limited in the embodiment of the present application.

The above two renderings are performed to obtain the simulated environment image and the marked image. The difference between the two renderings is that the first rendering is used to generate the simulated environment image obtained by the camera shooting the simulated driving environment. Specifically, according to the camera parameters The position information, orientation information and field of view information of the camera are used to determine the position, orientation, and shooting range (field of view) of the camera in the built three-dimensional simulation scene (ie, simulated driving environment), and the camera’s perspective The three-dimensional simulation scene is photographed to obtain the target image, and then the target image is adjusted according to the instructions of the rendering mode in the camera parameters such as resolution change, stretching rotation change, and/or color scale brightness change, so as to obtain the aforementioned simulation environment image . The second rendering is used to highlight the marked image elements in the simulated environment image (especially the drivable area) to obtain a marked image that is easy to read and understand. For example, as shown in Figure 2, the simulated simulated image is highlighted with a box The various image elements in the.

In one embodiment, the aforementioned motion module 110 is used to simulate the simulated driving process of the mobile platform in a simulated driving environment, and to generate spatial position information of the mobile platform during the simulated driving process. The aforementioned camera module 120 is used to simulate the driving process according to the simulated driving process. The spatial position information in generates the camera parameters of the mobile platform's camera in the simulation driving process. Specifically, when the mobile platform is driving in a simulated driving environment, the spatial position is constantly changing. Therefore, when the aforementioned motion module 110 simulates the movement of the mobile platform, it will generate the spatial position information of the mobile platform and transmit the spatial position information to The camera module 120, the spatial position information includes the position information and steering information of the mobile platform, etc. After obtaining the spatial position information of the mobile platform, the camera module 120 generates the camera parameters of the camera on the mobile platform according to the spatial position information. Specifically, directly Obtain the camera parameters of the camera corresponding to the spatial position information of the mobile platform from the correspondence table in the database, or except that the rendering mode and field angle information of the camera are preset, the position information and steering information in the camera parameters can be Calculate according to the calculation rules between the spatial position information of the mobile platform and the camera parameters of the camera. For example, calculate the position information of the camera according to the position of the mobile platform. Generally speaking, the relative position of the mobile platform and the camera is fixed. When the position information of the mobile platform is determined, the position information of the camera can be calculated according to the position information of the mobile platform and the relative position of the mobile platform and the camera. In addition, the camera generally collects images in the moving direction of the mobile platform, so the camera and The turning of the mobile platform is consistent. After determining the turning information of the mobile platform, the turning information of the mobile platform can be used as the turning information of the camera. Among them, different spatial position information can correspond to different rendering modes and field angle information, and the correspondence between the spatial position information and the rendering mode and field angle information in the camera parameters is stored in the database.

It can be seen that the embodiment of the present application can simulate the position change of the mobile platform when driving in the simulated driving environment, and determine the camera parameters of the camera according to the spatial position information of the mobile platform, so as to determine the camera parameters corresponding to the spatial position information, The camera captures the images that can be captured in a simulated driving scene.

In one embodiment, the above-mentioned rendering module 130 is used to determine the front view of the road obtained by the camera of the mobile platform during the simulated driving to photograph the simulated driving environment according to the camera parameters in the simulated driving process, and then perform image transformation on the front view of the road. Obtain the road top view, and use the road top view as the simulation environment image. Because in the actual automatic driving process, the camera generally captures the front view of the road, and the front view of the road is not conducive to the extraction of the drivable area, so the road front view can be image transformed to obtain the road top view, which is not only good for driving. Area detection is also more intuitive. Among them, the front view of the road is when the lens of the camera is facing the direction of the mobile platform, and the camera captures the image obtained by simulating the driving environment (equivalent to the road situation that the driver can see when the driver is looking ahead), and the top view of the road is the camera's When the lens is perpendicular to the traveling direction of the mobile platform, the camera will shoot the image obtained from the simulated traveling environment (equivalent to the bird's-eye image of the road when the helicopter is aerially photographed).

The above-mentioned image transformation of the road front view refers to the affine transformation of the road front view (also called perspective transformation), that is, the road front view is transformed into a road top view through a transformation matrix, and the transformation matrix is used to indicate the road front view and the road Rules of transformation between top views. It should be noted that the transformation matrices of cameras with different camera parameters may be different. The correctness of the transformation matrix affects the correctness of the affine transformation and indirectly affects the correctness of the driving area detection.

In the real world, the transformation matrix needs to be determined through calibration experiments, and this automatic driving simulation system can shoot the simulated driving environment at any angle and position by changing the camera parameters of the camera, so the front view of the road can be easily obtained at the same time And the top view of the road, and then calculate the above conversion matrix according to the front view of the road and the top view of the road, and compare the conversion matrix simulated in the automatic driving simulation system with the conversion matrix measured in the calibration experiment. The conversion matrix is verified by algorithm to judge the correctness of the conversion matrix, or the conversion matrix is adjusted appropriately to make the conversion matrix more correct, so this automatic driving simulation system can provide a test environment for automatic driving technologies such as driving area detection technology. In addition, the automatic driving simulation system can also replace the calibration experiment to obtain the conversion matrix corresponding to different camera parameters. Similarly, it is not difficult to think that this automatic driving simulation system can also be used to provide a test environment for other automatic driving technologies.

In one embodiment, the marking module 140 is used to mark the category of the image element in the simulation environment image to obtain a template cache containing the position information and category information of the image element of the simulation environment image. The rendering module 130 uses According to the template cache, the aforementioned simulation environment image is rendered to obtain the aforementioned marked image. The embodiment of the present application describes the process of using the marking module 140 and the rendering module 130 to mark and render the simulated environment image respectively, so as to finally obtain the marked image. The marking module 140 first identifies each image element in the rendering module 130 to obtain The category of each image element, and use the mark character to mark the category of the image element, and then transfer the mark character of the image element on the simulation environment image to the cache template, that is, according to the location of the image element on the simulation cache image, in the cache Mark the corresponding position of the module with marked characters, and then the rendering module 130 reads the position information and category information recorded in the cache template, determines the position and category of each image element in the simulation environment image, and renders the simulation environment image , To highlight the image elements in the simulation environment image, for example, use a box to mark the image elements in the simulation environment image, as shown in Figure 2. Wherein, the marking module 140 contains marking rules. When marking the simulation environment image, the marking module 140 first recognizes the category of the image element, and then obtains the marking character corresponding to the category of the image element in the marking rule. It can be any combination of characters, numbers, etc., used to uniquely determine the category of the image element, and different categories correspond to different marked characters.

In fact, the cache template is equivalent to a simplified simulation environment image. The size of the template cache is the same as that of the simulation environment image. Each image element in the simulation environment image has a mark character at the corresponding position on the cache template. Contains the position information and marked characters of each image element of the simulation environment image. When the cache module and the simulation environment image are combined, the position of the marked characters in the cache module completely overlaps with the image elements on the simulation environment image.

When the above-mentioned marking module 140 recognizes image elements in the simulated environment image, it at least recognizes the drivable area in the simulated environment image. Specifically, the simulated environment image is binarized according to the gray value, and then edge detection is used to obtain the lane line. In order to extract the detected lane lines, the detection method can detect multiple lane lines at the same time.

In one embodiment, the marking module 140 is used to analyze and recognize the simulated environment image, and assign a mark value to the image element according to the type of the image element in the simulated environment image identified by the analysis, where the image element contains obstacles. The image element and the image element of the drivable area, the image element of the obstacle and the image element of the drivable area have different label values. The above-mentioned marking characters are marked values (numerical values), and the image elements include image elements of the drivable area and image elements of obstacles. Among them, the marking characters of the drivable area are 0, including roads and road surfaces, and the marking characters of obstacles are 220, including buildings, road guardrails, driving cars and pedestrians, etc.

For example, assuming that the mark character of the drivable area is 0, the marking module 140 recognizes that pixels 1 to n in the simulated environment image are the drivable area, and then inputs the recognized result to the cache module, and the cache template The pixel point 1 to pixel point n are assigned a value of 0, and then the rendering module 130 reads the value of pixel point 1 to pixel point n in the buffer template to determine that pixel point 1 to pixel point n in the simulation environment image are Driveable area, and when rendering the simulated environment image, mark pixel 1 to pixel n in the simulated environment image as the driveable environment, for example, use a box to frame pixel 1 to pixel n, and A text label marking the drivable area.

In one embodiment, the image elements of the obstacles include image elements of static obstacles and image elements of dynamic obstacles, and the image elements of the static obstacles and the image elements of dynamic obstacles have different label values. The above obstacles are further divided into static obstacles and dynamic obstacles. Among them, static obstacles are marked with 200 characters, including buildings and road guardrails, and dynamic obstacles are marked with 255 characters, including moving cars. And pedestrians.

In one embodiment, the aforementioned rendering module 130 is configured to output the image elements of the simulated environment image according to the color indicated by the corresponding tag value to obtain the aforementioned tagged image. When the rendering module 130 renders the simulated environment image, it refers to outputting the image elements in the simulated environment image according to the color represented by the mark value, so as to obtain the mark image. Specifically, according to the position information and marking characters of the image elements contained in the above-mentioned cache module, the simulation environment image is output according to the position indicated by the position information as the color indicated by the marking characters corresponding to the position information, that is, where the image element is The position output is the color indicated by the label value of the image element, or the translucent color indicated by the label value of the image element is overlaid on the position of the image element. For example, the mark value of the drivable area is 0, the mark character of a static obstacle is 200, and the mark character of a dynamic obstacle is 255. In the red green blue (RGB, red green blue) color mode, the mark value 0 represents The RGB value (0, 0, 0) is represented as black, the marked value 200 represents the RGB value (200, 200, 200), which is represented as gray, and the marked value 255 represents the RGB value (255, 255, 255), which is represented as white, Then output the drivable area as black represented by the color value 0, or cover the drivable area with a layer of translucent black, and output the stationary obstacle as gray represented by the color value 200, or on the stationary obstacle Cover a layer of translucent gray, and output the dynamic obstacle as white represented by the color value of 255, or cover the dynamic obstacle with a layer of translucent white. The embodiment of the present application does not limit the color mode. In different color modes, the same label value may correspond to different colors, but in the same color mode, the label value and the color correspond one-to-one.

It can be seen that the automatic driving simulation system of the present application can simulate a real automatic driving simulation scene. Specifically, the movement module 110 is used to simulate the driving process of the mobile platform in the simulated driving environment, and then the camera module 120 and The rendering module 130 simulates the simulated environment image obtained by the camera shooting the simulated driving environment, and finally uses the marking module 140 and the rendering module 130 to detect the driving area of the simulated environment image, thereby obtaining the marked image for assisting automatic driving in path planning. It can be seen that this application can simulate a real driving scene and test the driving area detection technology, which provides a reliable algorithm verification for the driving area detection technology. In addition, since the system simulates a real driving scene, the The system can also be used to test more autonomous driving technologies in addition to driving area detection technologies.

It is understandable that the system architecture and business scenarios described in the embodiments of this application are intended to more clearly illustrate the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided in the embodiments of this application. Those of ordinary skill in the art It can be seen that with the evolution of the system architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of the present application are equally applicable to similar technical problems.

Based on the above description, the embodiment of the present invention proposes an automatic driving simulation method in FIG. 3, and the automatic driving simulation method can be implemented by the rendering module 130 in the aforementioned automatic driving simulation system.

In S301, obtain the camera parameters of the mobile platform's camera during the simulated driving process. The simulated driving process is the process of the simulated mobile platform driving in the simulated driving environment, and the simulated driving environment is to imitate the driving environment of the mobile platform in the real world. The constructed three-dimensional simulation environment, the camera parameters include at least one of the camera's position information, orientation information, rendering mode, and field of view information. The position information in the camera is the position and orientation information of the camera in the aforementioned simulated driving environment It is the shooting direction of the camera, the rendering mode is the adjustment method of the image, including resolution change, stretch rotation change and/or color scale brightness change, etc. The angle of view (fov, angle of view) is the angle range of the camera that can receive the image , Also known as the field of view, is different from the imaging range (angle of coverage), the field of view describes the image angle that the camera lens can capture.

It should be noted that the camera parameters of the above-mentioned camera are in line with the real data information of the real world, so in the real world the imaging of the camera can be determined according to the above-mentioned camera parameters, and the movement process of the above-mentioned mobile platform in the simulated driving environment is also in line with the real According to the physical rules of the world, the motion information of the interaction between the mobile platform and the simulated driving environment is provided by the physics engine. The physics engine essentially instructs the motion module 110 to follow the calculation rules when simulating the motion process of the mobile platform, and the motion simulated by the physics engine The process conforms to the physical rules in the real world. Among them, the movement information includes, for example, the spatial position information of the mobile platform, etc. The spatial position information includes the position information and steering information of the mobile platform.

In one implementation, before simulating the above-mentioned driving process, a simulated driving environment containing real-world three-dimensional things is constructed first. The simulated driving environment includes three-dimensional objects such as terrain, vegetation, weather systems, buildings, and roads. Specifically, the 3D model is constructed according to the size ratio of the 3D object in the real world, and then the 3D model is beautified, so that in addition to the shape, the color and pattern are closer to the object in the real world, and finally an object that can be rotated at will is obtained. And the realistic 3D model displayed at any angle, such as constructing the 3D model of the building with the scale of the building, and then overlaying the color graphics of the building on the 3D model, and lighting and adding shadows to the 3D model to obtain An architectural model that is very similar to a real-world building. By rotating the architectural model, the architectural model can be observed from any perspective.

In one embodiment, obtaining the camera parameters of the mobile platform's camera during the simulation driving process refers to obtaining the camera parameters of the mobile platform's camera during the simulation driving process generated according to the spatial position information of the mobile platform during the simulation driving process. Among them, the above-mentioned spatial position information of the mobile platform during the simulated driving refers to the spatial position information generated by the mobile platform driving in the simulated driving environment, including the position information and steering information of the mobile platform. The above-mentioned camera parameters of the mobile platform’s camera generated during the simulation driving process based on the spatial position information of the mobile platform during the simulation driving process refer to the camera parameters corresponding to the spatial position information of the mobile platform acquired in the correspondence table in the database. Camera parameters, or in addition to the camera's rendering mode and field angle information are preset, the position information and steering information in the camera parameters are calculated according to the calculation law between the spatial position information of the mobile platform and the camera parameters of the camera For example, the position information of the camera is calculated according to the position of the mobile platform. Generally speaking, the relative position of the mobile platform and the camera is fixed, so when the position information of the mobile platform is determined, it can be based on the position information of the mobile platform and the mobile platform. The relative position of the camera is used to calculate the position information of the camera. In addition, the camera generally collects images in the moving direction of the mobile platform, so the steering of the camera and the mobile platform are consistent. After the steering information of the mobile platform is determined, the mobile platform can be The turning information of the camera is used as the turning information of the camera. Among them, different spatial position information can correspond to different rendering modes and field angle information, and the correspondence between the spatial position information and the rendering mode and field angle information in the camera parameters is stored in the database.

In S302, according to the aforementioned camera parameters during the simulated driving process, it is determined that the camera of the mobile platform during the simulated driving process captures the simulated environment image obtained by the simulated driving environment. Specifically, according to the position information, orientation information, and field of view information in the camera parameters, the position, orientation, and shooting range (field of view) of the camera in the constructed three-dimensional simulation scene (ie, simulated driving environment) are respectively determined. , And shoot the 3D simulation scene from the camera's perspective to obtain the target image, and then adjust the target image resolution, stretch rotation, and/or color scale brightness changes according to the instructions of the rendering mode in the camera parameters , So as to obtain the above simulation environment image.

In one embodiment, the above-mentioned determination of the simulated environment image obtained by the camera of the mobile platform during the simulated driving to capture the simulated driving environment based on the camera parameters during the simulated driving process refers to the determination of the camera parameters in the simulated driving process. During the simulated driving process, the camera of the mobile platform shoots the front view of the road obtained from the simulated driving environment, and then performs image transformation on the front view of the road to obtain the road top view, and the road top view is used as the simulation environment image. Because in the actual automatic driving process, the camera generally captures the front view of the road, and the front view of the road is not conducive to the extraction of the drivable area, so the road front view can be image transformed to obtain the road top view, which is not only good for driving. Area detection is also more intuitive. Among them, the front view of the road is when the lens of the camera is facing the direction of the mobile platform, and the camera captures the image obtained by simulating the driving environment (equivalent to the road situation that the driver can see when the driver is looking ahead), and the top view of the road is the camera's When the lens is perpendicular to the traveling direction of the mobile platform, the camera will shoot the image obtained from the simulated traveling environment (equivalent to the bird's-eye image of the road when the helicopter is aerially photographed).

In S303, a marked simulation environment image is obtained. The marked simulated environment image means that each image element in the simulated environment image is marked, especially the drivable area is marked. The image elements include drivable areas, pedestrians, buildings, greenery, roads, etc. The image of the object. Among them, the drivable area is also called Freespace, which is a road on which the mobile platform can travel in automatic driving, and is used to provide path planning for automatic driving to avoid obstacles. The drivable area can be the entire road surface of the road, or a part of the road surface that contains key information of the road (such as road direction information, road midpoint information, etc.). More specifically, the drivable area includes structured pavement, semi-structured pavement, and unstructured pavement. Structured pavement is a pavement with a single pavement structure and road edges, such as urban main roads, high-speed roads, and national roads. , Provincial roads, etc., semi-structured roads are roads with various structures, such as parking lots, squares, etc., and unstructured roads are natural roads without structural layers, such as undeveloped uninhabited areas.

In S304, image rendering is performed on the marked simulated environment image to obtain a marked image. Specifically, highlight the marked image elements (especially the drivable area) in the simulated environment image to obtain easy-to-read and understandable marked images. The marked images are used to assist the mobile platform in path planning during automatic driving to avoid Obstacles, for example, as shown in Figure 2, use boxes to focus on each image element in the simulated image.

In one implementation, the aforementioned acquisition of the marked simulation environment image refers to the acquisition of the template cache containing the position information and category information of the image elements of the simulation environment image, and correspondingly, the aforementioned image rendering of the marked simulation environment image , Obtaining the marked image refers to rendering the simulation environment image according to the template cache to obtain the marked image. Among them, the size of the cache template is the same as that of the simulation environment image, which corresponds to the location of the image element on the simulation cache image. The corresponding position of the cache module is marked with the mark character of the image element, and the mark character represents the category to which the image element belongs. The cache template contains the location information and category information of each image element in the simulation environment image. Specifically, after obtaining the above-mentioned cache template, the simulation cache image is rendered according to the cache template, that is, while reading the position information and category information recorded in the cache template, the position and location of each image element in the simulation environment image are determined. Category, while rendering the simulation environment image to highlight the image elements in the simulation environment image, for example, using a box to mark the image elements in the simulation environment image, as shown in Figure 2.

It should be noted that the mark character is used to indicate the category of the image element. The mark character can be any combination of characters, numbers, etc., and is used to uniquely determine the category of the image element. Image elements of the same category correspond to the same mark character, and different categories The image elements correspond to different marked characters.

In one embodiment, the above-mentioned marked characters are marked values (numerical values), and the image elements in the simulated environment image after the above-mentioned marking are assigned marked values according to categories, where the image elements include the image elements of the obstacles and the driving area. The label values of the image elements, the image elements of the obstacles and the image elements of the drivable area are not the same. Among them, the marking character of the drivable area is 0, including roads and pavements, and the marking character of obstacles is 220, including buildings, road guardrails, driving cars, and pedestrians.

For example, suppose that the mark character of the drivable area is 0, and the pixel 1 to pixel n in the simulated environment image are the drivable area, then the pixel 1 to pixel n in the cache template are assigned the value 0, and then read Take the values from pixel 1 to pixel n in the cache template to determine that pixel 1 to pixel n in the simulation environment image are the drivable area, and when rendering the simulation environment image, use the simulation environment image Pixels 1 to n in are marked as a drivable environment. For example, use a box to frame pixel 1 to pixel n and mark the text label of the drivable area, as shown in Figure 2.

In one embodiment, the image elements of the simulated environment image are output according to the color indicated by the corresponding label value to obtain the above-mentioned label image. Specifically, according to the position information and marking characters of the image elements contained in the above-mentioned cache module, the simulation environment image is output according to the position indicated by the position information as the color indicated by the marking characters corresponding to the position information, that is, where the image element is The position output is the color indicated by the label value of the image element, or the translucent color indicated by the label value of the image element is overlaid on the position of the image element. For example, the mark value of the drivable area is 0, the mark character of a static obstacle is 200, and the mark character of a dynamic obstacle is 255. In the red green blue (RGB, red green blue) color mode, the mark value 0 represents The RGB value (0, 0, 0) is represented as black, the marked value 200 represents the RGB value (200, 200, 200), which is represented as gray, and the marked value 255 represents the RGB value (255, 255, 255), which is represented as white, Then output the drivable area as black represented by the color value 0, or cover the drivable area with a layer of translucent black, and output the stationary obstacle as gray represented by the color value 200, or on the stationary obstacle Cover a layer of translucent gray, and output the dynamic obstacle as white represented by the color value of 255, or cover the dynamic obstacle with a layer of translucent white. The embodiment of the present application does not limit the color mode. In different color modes, the same label value may correspond to different colors, but in the same color mode, the label value and the color correspond one-to-one.

It can be seen that the embodiment of the present application can simulate the simulated environment image obtained by shooting the simulated driving environment by the camera of the mobile platform according to the camera parameters, and then render the simulated environment image after marking to obtain a path planning aid for automatic driving Mark the image. It can be seen that the embodiments of the present application can be used to test the driving area detection technology, and provide a reliable algorithm verification for the driving area detection technology.

Based on the embodiment of the previous application, the embodiment of the present invention also proposes a more detailed automatic driving simulation method, as shown in FIG. 4.

In S401, the camera parameters of the camera of the mobile platform during the simulation driving, which are generated according to the spatial position information of the mobile platform during the simulation driving, are acquired. Among them, the simulated driving process is the process of the simulated mobile platform driving in the simulated driving environment, and the simulated driving environment is a three-dimensional simulation environment constructed following the driving environment of the mobile platform in the real world. The above-mentioned spatial position information is the simulated driving environment of the mobile platform. The position information and steering information during the driving process. The camera parameters include at least one of the camera’s position information, orientation information, rendering mode, and field of view information. The position information in the camera is determined by the camera in the aforementioned simulated driving environment. The position, the orientation information is the shooting direction of the camera, the rendering mode is the adjustment method of the image, including resolution change, stretching rotation change and/or color scale brightness change, etc., the angle of view (fov, angle of view) is the camera The range of angles that can receive images, also known as the field of view, is different from the imaging range (angle of coverage). The field of view describes the image angle that the camera lens can capture.

The above-mentioned camera parameters of the mobile platform’s camera generated during the simulation driving process based on the spatial position information of the mobile platform during the simulation driving process refer to the camera parameters corresponding to the spatial position information of the mobile platform acquired in the correspondence table in the database. Camera parameters, or in addition to the camera's rendering mode and field angle information are preset, the position information and steering information in the camera parameters are calculated according to the calculation law between the spatial position information of the mobile platform and the camera parameters of the camera For example, the position information of the camera is calculated according to the position of the mobile platform. Generally speaking, the relative position of the mobile platform and the camera is fixed, so when the position information of the mobile platform is determined, it can be based on the position information of the mobile platform and the mobile platform. The relative position of the camera is used to calculate the position information of the camera. In addition, the camera generally collects images in the moving direction of the mobile platform, so the steering of the camera and the mobile platform are consistent. After the steering information of the mobile platform is determined, the mobile platform can be The turning information of the camera is used as the turning information of the camera. Among them, different spatial position information can correspond to different rendering modes and field angle information, and the correspondence between the spatial position information and the rendering mode and field angle information in the camera parameters is stored in the database.

In S402, according to the aforementioned camera parameters during the simulated driving process, it is determined that the camera of the mobile platform during the simulated driving process captures the road front view obtained by the simulated driving environment. Specifically, according to the position information, orientation information, and field of view information in the camera parameters, the position, orientation, and shooting range (field of view) of the camera in the constructed three-dimensional simulation scene (ie, simulated driving environment) are respectively determined. , And shoot the 3D simulation scene from the camera's perspective to obtain the target image, and then adjust the target image resolution, stretch rotation, and/or color scale brightness changes according to the instructions of the rendering mode in the camera parameters To get a front view of the road.

In S403, image transformation is performed on the above-mentioned road front view to obtain a road top view, and the road top view is used as a simulation environment image. Specifically, the image transformation of the front view of the road refers to the affine transformation of the front view of the road (also called perspective transformation), that is, the front view of the road is transformed into a top view of the road through a transformation matrix, and the transformation matrix is used to indicate the front view of the road. Conversion rules between and the top view of the road. It should be noted that the transformation matrices of cameras with different camera parameters may be different. The correctness of the transformation matrix affects the correctness of the affine transformation and indirectly affects the correctness of the driving area detection.

It should be noted that in the actual automatic driving process, the camera usually captures the front view of the road, and the front view of the road is not conducive to the extraction of the drivable area, so the road front view can be image transformed to obtain the road top view. The top view is not only conducive to driving area detection, but also more intuitive. Among them, the front view of the road is when the lens of the camera is facing the direction of the mobile platform, and the camera captures the image obtained by simulating the driving environment (equivalent to the road situation that the driver can see when the driver is looking ahead), and the top view of the road is the camera's When the lens is perpendicular to the traveling direction of the mobile platform, the camera will shoot the image obtained from the simulated traveling environment (equivalent to the bird's-eye image of the road when the helicopter is aerially photographed).

In S404, a template cache containing the location information and category information of the image elements of the aforementioned simulation environment image is obtained. Among them, the size of the cache template is the same as that of the simulation environment image, which corresponds to the location of the image element on the simulation cache image. The corresponding position of the cache module is marked with the mark character of the image element, and the mark character represents the category to which the image element belongs. The cache template contains the location information and category information of each image element in the simulation environment image. Among them, the image elements include images of various objects such as drivable areas, pedestrians, buildings, greenery and roads.

The above-mentioned drivable area is also called Freespace, which is a road on which the mobile platform can travel in automatic driving, and is used to provide path planning for automatic driving to avoid obstacles. The drivable area can be the entire road surface of the road, or a part of the road surface that contains key information of the road (such as road direction information, road midpoint information, etc.). More specifically, the drivable area includes structured pavement, semi-structured pavement, and unstructured pavement. Structured pavement is a pavement with a single pavement structure and road edges, such as urban main roads, high-speed roads, and national roads. , Provincial roads, etc., semi-structured roads are roads with various structures, such as parking lots, squares, etc., and unstructured roads are natural roads without structural layers, such as undeveloped uninhabited areas.

In S405, the simulation environment image is rendered according to the template cache to obtain a marked image. Specifically, after obtaining the above-mentioned cache template, the simulation cache image is rendered according to the cache template, that is, while reading the position information and category information recorded in the cache template, the position and location of each image element in the simulation environment image are determined. Category, while rendering the simulation environment image to highlight the image elements in the simulation environment image, to obtain a marked image that is used to assist the mobile platform in path planning to avoid obstacles during automatic driving, such as using a box to mark the simulation The image elements in the environment image are marked as shown in Figure 2.

For example, suppose that the mark character of the drivable area is 0, and the pixel 1 to pixel n in the simulated environment image are the drivable area, then the pixel 1 to pixel n in the cache template are assigned the value 0, and then read Take the values from pixel 1 to pixel n in the cache template to determine that pixel 1 to pixel n in the simulated environment image are the drivable area, and when the simulated environment image is rendered, the simulated environment image Pixels 1 to n in are marked as a drivable environment. For example, use a box to frame pixel 1 to pixel n and mark the text label of the drivable area, as shown in Figure 2.

The embodiment of the present application is more detailed than the embodiment of the previous application, and describes in detail the process of determining the mobile platform according to the camera parameters to capture the simulated driving environment during the driving process to obtain the simulated environment image. It should be noted that the above description of the various embodiments tends to emphasize the differences between the various embodiments, and the similarities or similarities can be referred to each other. For the sake of brevity, details are not repeated herein.

Based on the description of the foregoing method embodiment, an embodiment of the present application further provides an automatic driving simulation device, which is used to execute a unit of any one of the foregoing automatic driving simulation methods. Specifically, refer to FIG. 5, which is a schematic block diagram of an automatic driving simulation device provided by an embodiment of the present application. The automatic driving simulation device of the embodiment of the present application includes: an acquisition unit 510 and a rendering unit 520. specific:

The acquiring unit 510 is configured to acquire camera parameters of the camera of the mobile platform during the simulated driving process;

The rendering unit 520 is configured to determine, according to the camera parameters in the simulated driving process, that the camera of the mobile platform captures the simulated environment image obtained in the simulated driving environment during the simulated driving process;

The above-mentioned acquiring unit 510 is also used to acquire a marked simulation environment image;

The rendering unit 520 is also used to perform image rendering on the marked simulated environment image to obtain a marked image.

In an implementation, the acquisition unit 510 is specifically configured to acquire the camera parameters of the camera of the mobile platform during the simulation driving process generated according to the spatial position information of the mobile platform during the simulation driving process.

In an implementation, the rendering unit 520 is specifically configured to determine, according to the camera parameters during the simulation driving process, the front view of the road obtained by the camera of the mobile platform during the simulation driving process, which is obtained by shooting the simulation driving environment; The simulation device also includes a conversion unit 530, which is used to perform image conversion on the front view of the road to obtain a top view of the road and use the top view of the road as the simulation environment image.

In an implementation, the acquisition unit 510 is specifically configured to acquire a template cache containing the location information and category information of the image elements of the simulation environment image; the rendering unit is further configured to perform processing on the simulation environment image according to the template cache. Render to get the above-mentioned marked image.

In one implementation, the image elements in the marked simulated environment image are assigned marked values according to categories, wherein the image elements include the image elements of the obstacle and the image elements of the drivable area, and the image elements of the obstacle The label value of the image element in the above-mentioned drivable area is different.

In an implementation, the aforementioned rendering unit 520 is specifically configured to output the image elements of the aforementioned simulated environment image according to the color indicated by the corresponding tag value to obtain the aforementioned tagged image.

In one implementation, the image element of the obstacle includes the image element of the static obstacle and the image element of the dynamic obstacle, and the image elements of the static obstacle and the image element of the dynamic obstacle have different label values.

In an implementation, the aforementioned camera parameters include at least one of position information, orientation information, rendering mode, and field of view information of the camera.

A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing relevant hardware through a computer program. The above-mentioned program can be stored in a computer-readable storage medium. When executed, it may include the processes of the above-mentioned method embodiments. Among them, the aforementioned storage medium may be a magnetic disk, an optical disc, a read-only memory (Read-Only Memory, ROM), or a random access memory (Random Access Memory, RAM), etc.

It should be noted that, for the specific working process of the automatic driving simulation device described above, reference may be made to the related descriptions in the foregoing embodiments, which will not be repeated here.

Refer to FIG. 6, which is a structural block diagram of an automatic driving simulation device provided by another embodiment of the present application. The automatic driving simulation device in this embodiment as shown in the figure may include: one or more processors 610 and a memory 620. The aforementioned processor 610 and memory 620 are connected through a bus 630. The memory 620 is configured to store a computer program, and the computer program includes program instructions, and the processor 610 is configured to execute the program instructions stored in the memory 620.

The processor 610 is configured to perform the functions of the obtaining unit 510, and is used to obtain the camera parameters of the camera of the mobile platform during the simulation driving process; and is also used to perform the functions of the rendering unit 520, and is used to perform the above-mentioned camera parameters during the simulation driving process. , Determining that the camera of the mobile platform captures the simulated environment image obtained from the simulated driving environment during the simulated driving process; is also used to obtain the marked simulated environment image; and is also used to perform image rendering on the marked simulated environment image, Get the marked image.

In an implementation, the processor 610 is specifically configured to obtain the camera parameters of the camera of the mobile platform during the simulation driving process generated according to the spatial position information of the mobile platform during the simulation driving process.

In an implementation, the processor 610 is specifically configured to determine, according to the camera parameters during the simulated driving process, that the camera of the mobile platform captures the road front view obtained by the simulated driving environment during the simulated driving process; 610 is also used to perform the function of the transformation unit 530, which is used to perform image transformation on the front view of the road to obtain a top view of the road, and use the top view of the road as the simulation environment image.

In an implementation, the aforementioned processor 610 is specifically configured to obtain a template cache containing the location information and category information of the image elements of the aforementioned simulated environment image; and is also configured to render the aforementioned simulated environment image according to the aforementioned template cache to obtain the aforementioned Mark the image.

In an implementation, the aforementioned processor 610 is specifically configured to output the image elements of the aforementioned simulated environment image according to the color indicated by the corresponding mark value to obtain the aforementioned marked image.

In one embodiment, the processor may be a central processing unit (Central Processing Unit, CPU), and the processor may also be another general-purpose processor, that is, a microprocessor or any conventional processor, such as digital signal processing DSP (Digital Signal Processor), Application Specific Integrated Circuit (ASIC), Field-Programmable Gate Array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware Components, etc.

The memory 620 may include a read-only memory and a random access memory, and provides instructions and data to the processor 610. A part of the memory 620 may also include a non-volatile random access memory. For example, the memory 620 may also store device type information.

In specific implementation, the processor 610 described in the embodiment of this application can execute the implementation described in the first and second embodiments of the anchor moving reminder method provided in the embodiment of this application, and can also execute the implementation of this application. The implementation of the automatic driving simulation device described in the example will not be repeated here.

In another embodiment of the present application, a computer-readable storage medium is provided. The computer-readable storage medium stores a computer program, the computer program includes program instructions, and the program instructions are executed by a processor.

The computer-readable storage medium may be an internal storage unit of the automatic driving simulation device of any of the foregoing embodiments, such as the hard disk or memory of the automatic driving simulation device. The computer-readable storage medium can also be an external storage device of the automated driving simulation device, such as a plug-in hard disk, a smart media card (SMC), and a secure digital (SD) card equipped on the automated driving simulation device. , Flash Card, etc. Further, the computer-readable storage medium may also include both an internal storage unit of the automatic driving simulation device and an external storage device. The computer-readable storage medium is used to store computer programs and other programs and data required by the automatic driving simulation device. The computer-readable storage medium can also be used to temporarily store data that has been output or will be output.

The above-disclosed are only part of the embodiments of the present invention. Of course, the scope of rights of the present invention cannot be limited by this. Those of ordinary skill in the art can understand all or part of the processes for implementing the above-mentioned embodiments and make them in accordance with the claims of the present invention. The equivalent changes still fall within the scope of the invention.

Claims

An automatic driving simulation system is characterized in that it comprises:

The motion module is used to simulate the simulated driving process of the mobile platform in the simulated driving environment;

The camera module is used to determine the camera parameters of the camera of the mobile platform during the simulation driving process;

A rendering module, configured to determine, according to the camera parameters during the simulated driving process, that the camera of the mobile platform captures the simulated environment image obtained by the simulated driving environment during the simulated driving process;

A marking module, configured to mark the simulated environment image, where at least the drivable area in the simulated environment image is marked when marking;

The rendering module is also used to perform image rendering on the marked simulated environment image to obtain a marked image.
The system according to claim 1, wherein:

The motion module is used to simulate a simulated driving process of the mobile platform in a simulated driving environment, and generate spatial position information of the mobile platform during the simulated driving process;

The camera module is configured to generate camera parameters of the camera of the mobile platform during the simulation driving process according to the spatial position information during the simulation driving process.
The system according to claim 1, wherein the rendering module is used for:

Determine, according to the camera parameters during the simulated driving process, that the camera of the mobile platform captures the road front view obtained by the simulated driving environment during the simulated driving process;

Performing image transformation on the front view of the road to obtain a top view of the road;

Use the road top view as the simulation environment image.
The system according to claim 1, wherein:

The marking module is used to mark the category of the image element in the simulation environment image to obtain a template cache containing the position information and category information of the image element of the simulation environment image;

The rendering module is configured to render the simulation environment image according to the template cache to obtain the marked image.
The system according to claim 1, wherein the marking module is configured to analyze and recognize the simulation environment image, and provide the image according to the classification of the image element in the simulation environment image identified by the analysis. The element is assigned a label value, wherein the image element includes the image element of the obstacle and the image element of the drivable area, and the label value of the image element of the obstacle and the image element of the drivable area are different.
The system according to claim 5, wherein the rendering module is configured to output the image elements of the simulated environment image according to the color indicated by the corresponding label value to obtain the label image.
The system according to claim 5, wherein the image element of the obstacle includes the image element of the static obstacle and the image element of the dynamic obstacle, the image element of the static obstacle and the image element of the dynamic obstacle The tag values of the image elements are not the same.
The system according to any one of claims 1 to 7, wherein the camera parameters include at least one of position information, orientation information, rendering mode, and field of view information of the camera.
An automatic driving simulation method, characterized by comprising:

Obtain the camera parameters of the mobile platform's camera in the simulation driving process;

Determining, according to the camera parameters in the simulated driving process, the simulated environment image obtained by the camera of the mobile platform during the simulated driving process to capture the simulated driving environment;

Obtain the marked simulation environment image;

Image rendering is performed on the marked simulated environment image to obtain a marked image.
The method according to claim 9, wherein said acquiring camera parameters of a camera of a mobile platform during a simulated driving process comprises:

Obtain the camera parameters of the camera of the mobile platform during the simulation driving process generated according to the spatial position information of the mobile platform during the simulation driving process.
The method according to claim 9, characterized in that said determining the simulation environment obtained by the camera of the mobile platform during the simulation driving photographing the simulated driving environment according to the camera parameters in the simulation driving process Images, including:

Determine, according to the camera parameters during the simulated driving process, that the camera of the mobile platform captures the road front view obtained by the simulated driving environment during the simulated driving process;

Performing image transformation on the front view of the road to obtain a top view of the road;

Use the road top view as the simulation environment image.
The method according to claim 9, wherein said acquiring the marked simulated environment image comprises:

Acquiring a template cache containing location information and category information of image elements of the simulation environment image;

The performing image rendering on the marked simulated environment image to obtain the marked image includes:

Render the simulation environment image according to the template cache to obtain the marked image.
The method according to claim 9, wherein the image elements in the marked simulated environment image are assigned marked values according to categories, wherein the image elements include image elements of obstacles and areas of the drivable area. Image elements, the image elements of the obstacle and the image elements of the drivable area have different label values.
The method according to claim 13, wherein the performing image rendering on the marked simulated environment image to obtain the marked image comprises:

The image elements of the simulation environment image are output according to the color indicated by the corresponding label value to obtain the label image.
The method according to claim 13, wherein the image element of the obstacle includes the image element of the static obstacle and the image element of the dynamic obstacle, the image element of the static obstacle and the image element of the dynamic obstacle The tag values of the image elements are not the same.
The method according to any one of claims 9 to 15, wherein the camera parameters include at least one of position information, orientation information, rendering mode, and field of view information of the camera.
An automatic driving simulation device, characterized by comprising:

The acquiring unit is used to acquire the camera parameters of the camera of the mobile platform during the simulated driving process;

A rendering unit, configured to determine, according to the camera parameters during the simulated driving process, that the camera of the mobile platform captures the simulated environment image obtained by the simulated driving environment during the simulated driving process;

The acquiring unit is also used to acquire a marked simulation environment image;

The rendering unit is also configured to perform image rendering on the marked simulated environment image to obtain a marked image.
An automatic driving simulation device, characterized by comprising a processor and a memory, the processor and the memory are connected to each other, wherein the memory is used to store a computer program, the computer program includes program instructions, and the processor is It is configured to call the program instructions to execute the method according to any one of claims 9-16.
A computer-readable storage medium, wherein the computer storage medium stores a computer program, the computer program includes program instructions, and the program instructions are executed by a processor to execute any one of claims 9-16 The method described in the item.