WO2022141294A1

WO2022141294A1 - Simulation test method and system, simulator, storage medium, and program product

Info

Publication number: WO2022141294A1
Application number: PCT/CN2020/141798
Authority: WO
Inventors: 张树汉; 刘天博; 应佳行
Original assignee: 深圳市大疆创新科技有限公司
Priority date: 2020-12-30
Filing date: 2020-12-30
Publication date: 2022-07-07
Also published as: CN114846515A

Abstract

A simulation test method and system, a simulator, a storage medium, and a program product. The method comprises: obtaining a control signal for a movable platform output by a control system; on the basis of the control signal, simulating movement of the movable platform in a scene model to obtain a relative pose between the movable platform and a scene element; generating a plurality of scene images according to the relative pose, the plurality of scene images comprising scene images observed by at least two of vision sensors when the movable platform moves in the scene model; and outputting the plurality of scene images, so that the control system generates a corresponding control signal according to the plurality of scene images. Thus, actual image input of the control system can be simulated, a simulation test of the control system based on stereoscopic vision is implemented, the test efficiency is effectively improved, and the test cost is reduced.

Description

Simulation test method, system, simulator, storage medium and program product

technical field

The embodiments of the present application relate to the technical field of visual processing, and in particular, to a simulation testing method, system, simulator, storage medium, and program product.

Background technique

With the continuous advancement of image processing technology, intelligent control systems based on visual images, such as automatic driving systems, are also constantly developing. The intelligent control system based on visual images can perform corresponding operations through preset algorithms according to the images collected by the camera, so as to realize functions such as automatic driving and provide convenience for users.

In order to ensure the performance of the system, the system needs to be tested during the product development and verification stage of the system. For the intelligent control system of the present application based on stereo vision, it is difficult to realize the test of the system through the conventional visual simulation technology, and can only rely on the actual road test, the test efficiency is low, and the cost is high.

SUMMARY OF THE INVENTION

Embodiments of the present application provide a simulation testing method, system, simulator, storage medium and program product, which are used to implement testing of a control system based on stereo vision.

In the first aspect, an embodiment of the present application provides a simulation test method, which is applied to a simulation test system; the simulation test system is used to test a control system of a movable platform based on a virtual scene model, and the scene model includes multiple scene elements; the control system is configured to, based on scene images observed by at least two visual sensors, output a control signal for controlling the movable platform; the method includes:

acquiring the control signal of the movable platform output by the control system;

Based on the control signal, simulate the movement of the movable platform in the scene model to obtain the relative pose between the movable platform and the scene element;

generating a plurality of scene images according to the relative pose, the plurality of scene images including scene images observed by at least two of the vision sensors when the movable platform moves in the scene model;

The plurality of scene images are output, so that the control system generates corresponding control signals according to the plurality of scene images.

In a second aspect, an embodiment of the present application provides a simulation testing method, including:

obtaining a control signal output by the control system to be tested, where the control signal is a control signal for controlling the movable platform output by the control system according to the historically input scene image and a preset control model;

outputting at least two scene images according to the control signal and the virtual scene model;

Wherein, the scene model includes multiple scene elements;

The at least two scene images include a first scene image and a second scene image, the first scene image includes a first pixel area, the second scene image includes a second pixel area, the first pixel area and the The second pixel area describes the same scene element; the position of the first pixel area in the first scene image is between the position of the second pixel area in the second scene image The deviation is determined according to the relative pose between the visual sensors of the movable platform;

The outputted multiple scene images are used to represent the movement of the movable platform in response to the control signal, and after the relative pose between the movable platform and the scene elements is changed, observed by the vision sensor of the movable platform to the scene image of the scene element.

In a third aspect, an embodiment of the present application provides a simulation test system, where the simulation test system is used to test a control system of a movable platform based on a virtual scene model, where the scene model includes a plurality of scene elements;

The control system is configured to output a control signal for controlling the movable platform based on scene images observed by at least two visual sensors;

The simulation test system includes an emulator and an image output device;

The simulator is used to obtain the control signal of the movable platform output by the control system, and based on the control signal, simulate the movement of the movable platform in the scene model, and obtain the relationship between the movable platform and the relative poses between scene elements, and generate an observable scene image based on the relative poses;

The image output device is configured to acquire the scene image generated by the simulator, and output a plurality of scene images according to the acquired scene image, and the plurality of scene images include at least two images when the movable platform moves in the scene model. scene images observed by the visual sensor, so that the control system generates corresponding control signals according to the plurality of scene images.

In a fourth aspect, an embodiment of the present application provides a simulation test system, including: a simulator and an image output device;

The simulator is used to obtain the control signal output by the control system to be tested, and determine the corresponding scene image according to the control signal and the virtual scene model; wherein, the virtual scene model includes a plurality of scene elements;

The image output device is configured to acquire the scene image, and output at least two scene images according to the scene image;

Wherein, the virtual scene model includes a plurality of scene elements;

The control signal is a control signal for controlling the movable platform output by the control system according to the historically input at least two scene images and a preset control model;

In a fifth aspect, an embodiment of the present application provides an emulator, including: a memory and at least one processor;

the memory stores computer-executable instructions;

The at least one processor executes computer-executable instructions stored in the memory to cause the at least one processor to perform the method of the first aspect or the second aspect.

In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored; when the computer program is executed, the method described in the first aspect or the second aspect is implemented .

In a seventh aspect, an embodiment of the present application provides a computer program product, including a computer program, which implements the method described in the first aspect or the second aspect when the computer program is executed by a processor.

In the simulation test method, system, simulator, storage medium and program product provided by the embodiments of the present application, the simulation test system is used to test the control system of the movable platform based on a virtual scene model, and the scene model includes a plurality of scene elements, The control system is configured to output a control signal for controlling the movable platform based on scene images observed by at least two visual sensors, wherein the simulation testing system can specifically acquire the movable platform output by the control system. The control signal of the platform, based on the control signal, simulate the movement of the movable platform in the scene model, obtain the relative pose between the movable platform and the scene element, and generate the relative pose according to the relative pose A plurality of scene images, the plurality of scene images include scene images observed by at least two of the visual sensors when the movable platform moves in the scene model, and the plurality of scene images are output, so that The control system generates corresponding control signals according to the plurality of scene images, so that the actual image input of the control system can be simulated by generating scene images observable by at least two vision sensors, so as to realize the simulation of the control system based on stereo vision Test, test and verify the basic functions of the control system economically and efficiently, solve practical problems such as unreliability, incomplete scene test categories and high investment caused by relying on a large number of road tests, effectively improve test efficiency and reduce test costs.

Description of drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following will briefly introduce the accompanying drawings used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description These are some embodiments of the present application. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings without any creative effort.

1 is a schematic diagram of an application scenario provided by an embodiment of the present application;

2 is a schematic flowchart of a simulation testing method provided in an embodiment of the present application;

3A is a schematic flowchart of another simulation testing method provided by an embodiment of the present application;

3B is a schematic diagram of the positions of scene elements in a scene image according to an embodiment of the present application;

4 is a schematic flowchart of another simulation testing method provided by an embodiment of the present application;

5 is a schematic diagram of a test architecture based on multiple display devices provided by an embodiment of the present application;

6 is a schematic diagram of a calibration pattern provided by an embodiment of the present application;

7 is a schematic diagram of a test architecture based on an optical system provided by an embodiment of the present application;

FIG. 8 is a schematic diagram of a test architecture based on a 3D display device provided by an embodiment of the present application;

9 is a schematic diagram of a test architecture based on an image output interface provided by an embodiment of the present application;

10 is a schematic diagram of a test architecture based on a direct connection of an emulator provided by an embodiment of the present application;

FIG. 11A is a schematic structural diagram of a simulation testing device provided by an embodiment of the present application;

FIG. 11B is a schematic structural diagram of another simulation testing device provided by an embodiment of the present application;

12A is a schematic structural diagram of a simulation test system provided by an embodiment of the application;

12B is a schematic structural diagram of another simulation test system provided by an embodiment of the present application;

FIG. 13 is a schematic structural diagram of an emulator provided by an embodiment of the present application.

Detailed ways

In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments It is a part of the embodiments of the present application, but not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

The solutions provided by the embodiments of the present application can be applied to simulation testing of the control system of the movable platform. The movable platform can be any device that can move autonomously, such as a vehicle, a ship, an aircraft, an intelligent robot, and the like. The control system may be a control system based on visual images, and the control system can obtain images of the surrounding environment through the visual sensor of the movable platform, and output corresponding control signals according to the environment images, so as to realize the control of the movable platform.

In practical applications, simulation test verification is often essential for control systems based on visual images. Taking the driving control system of the vehicle as an example, in order to ensure the performance of the system, in the stage of product development and verification of the system, it is necessary to carry out simulation tests and actual road tests on the system. In a simulation test, a camera can be used as an image input to test the performance of the system in a simulated environment.

FIG. 1 is a schematic diagram of an application scenario provided by an embodiment of the present application. As shown in Figure 1, the driving control system and the camera connected to it constitute the tested system. The simulator can output road video to the display device, and the display device displays it. After the camera captures the road video displayed by the display device, The automatic driving system can output driving control signals according to the road video collected by the camera, and the simulator further adjusts the output road video according to the driving control signals.

In this way, the simulation test can play the road video to the camera for "viewing", so that the driving control system "thinks" that it is driving on the actual road, and then makes the same calculation as the actual road test and outputs the corresponding driving control signal to the simulator. The simulator then determines the next frame of road image through the control signal output by the driving control system and outputs it, so as to realize the test verification process.

The above test methods are generally called Hardware In Loop (HIL) experiments, which completely test the entire system (including the hardware and software parts of the system), in the process of automotive software process improvement and capability evaluation (Automotive Software The V-module described in Process Improvement and Capacity Determination (A-SPICE) belongs to the System Qualification Test level and is an indispensable part of the system verification process.

Through the above method, the simulation test of hardware-in-the-loop can be realized, so that the product developed by using the vision technology does not need to be tested in the actual use scene when performing functional verification. However, the driving control system based on a single camera does not have the problem of stereo depth, so the simulation test can be realized through a single display device. For the driving control system that relies on the stereo vision system as the observation input source, the obtained data can be obtained by means of a single display device. After the image is input to the system through two cameras, the stereoscopic scene described by the video stream cannot be reconstructed by stereo matching, because the image obtained by the two cameras is a plane, not an actual stereoscopic scene, and the depth information is lost in principle. As a result, the stereo information cannot be reconstructed twice. Therefore, the driving control system based on stereo vision is difficult to be tested by conventional visual simulation technology, and can only rely on the actual road test, which has low test efficiency and high cost.

In view of this, the embodiment of the present application provides a simulation test method, which can obtain the driving control signal output by the driving control system to be tested, and simulate the vehicle in the virtual scene model according to the driving control signal and the preset virtual scene model. and generate scene images that can be observed by multiple visual sensors of the vehicle, so that the driving control system can construct stereoscopic information according to the generated multiple scene images and output corresponding driving control signals, so as to realize driving based on stereo vision. Simulation test of the control system, so as to cost-effectively test and verify most of the basic functions of the driving control system in the simulation test. Improve test efficiency and reduce test cost.

Some embodiments of the present application will be described in detail below with reference to the accompanying drawings. The following embodiments and features in the embodiments may be combined with each other without conflict between the embodiments.

FIG. 2 is a schematic flowchart of a simulation testing method provided by an embodiment of the present application. The method in this embodiment can be applied to a simulation test system. The simulation test system can be used to test the control system of the movable platform based on a virtual scene model, and the scene model includes a plurality of scene elements. The control system may be configured to output control signals for controlling the movable platform based on images of the scene observed by at least two vision sensors.

The simulation testing system may include an emulator, and the emulator executes the following method, or may include an emulator and other devices, and the emulator and other devices jointly complete the following method. As shown in Figure 2, the method may include:

Step 201: Acquire the control signal of the movable platform output by the control system.

The embodiments of the present application can be applied to the simulation test of the control system of any type of movable platform. For ease of description, this embodiment is described by taking the test of the driving control system of the vehicle as an example. It can be understood that the method is also suitable for testing control systems of other movable platforms.

Correspondingly, the control system in this step may be a driving control system applied to a vehicle, the output control signal may be a driving control signal, and the movable platform may be a vehicle. The driving control system can be implemented based on software, hardware or a combination of software and hardware. In this step, the driving control signal output by the driving control system to be tested may be obtained, and the driving control signal is the driving control signal output by the driving control system according to the historically input scene image and the preset driving control model for controlling the vehicle. driving control signal. Wherein, the driving control system may be an automatic driving system (Autopilot System), an automatic driving assistance system (Automatic Driving Assistance System, ADAS) or any system capable of realizing driving control.

Specifically, the vehicle may be provided with multiple visual sensors, and the multiple visual sensors may be set at different positions of the vehicle, for example, may be set at different positions on the front windshield of the vehicle, and the multiple visual sensors at different positions may collect to different pictures, so that the depth information of the surrounding environment can be constructed through multiple pictures for better driving control.

The driving control model may be a preset driving control model, the input of the model may include images respectively collected by a plurality of visual sensors, and the output may include corresponding driving control signals. The driving control signal may be used to control the vehicle. Optionally, the driving control signal may be used to control the speed, direction, braking and other aspects of the vehicle.

The driving control signal is generated based on images collected by multiple visual sensors. For example, when it is determined that there is an obstacle ahead according to the images collected by multiple visual sensors, the output driving control signal may be: braking; When the image collected by each visual sensor determines that the road ahead is flat and free of obstacles, the output driving control signal can be: driving at acceleration a; and so on.

Optionally, the driving control signal may also be comprehensively determined on the basis of images collected by multiple visual sensors and in combination with other information such as route planning, road control information, and the like. For example, when it is determined according to images collected by multiple visual sensors that the traffic light ahead is currently in a green state, and a left turn is currently required according to the path planning, the output driving control signal may be: turn left.

In this embodiment of the present application, the movable platform may be a physical movable platform or a virtual movable platform, and the visual sensor may be a physical visual sensor or a virtual visual sensor.

For example, in the simulation test stage, it is not necessary to install the driving control system directly into the actual vehicle for vehicle testing. vehicle.

Optionally, the driving control system may be connected to a simulator, the simulator may output scene images to the driving control system, and the driving control system may output driving control signals to the simulator according to historically input scene images and a preset driving control model. , and then the simulator updates the scene image according to the driving control signal and outputs it to the driving control system.

Specifically, the simulator can update the position of the vehicle in the virtual scene in real time according to the driving control signal, and then update the scene image that the driving control system can "view". When the vehicle moves forward or backward, the scene image will also change accordingly, similar to a racing game, which is equivalent to the driving control system controlling the vehicle to run in a virtual scene.

Step 202: Based on the control signal, simulate the movement of the movable platform in the scene model to obtain the relative pose between the movable platform and the scene element.

Wherein, the scene element can be any element in the scene model, for example, during the road simulation test, the scene element can include lane lines, roadblocks, trees, pedestrians, traffic lights, etc., for simulating the actual road surroundings.

The relative pose may include position and/or angle information of the movable platform relative to one or more scene elements in the scene model.

Step 203: Generate a plurality of scene images according to the relative poses, where the plurality of scene images include scene images observed by at least two of the visual sensors when the movable platform moves in the scene model.

Among them, at least two vision sensors constitute a multi-eye vision sensor, which can collect images of the surrounding environment. The embodiments of the present application do not limit the number and pose of the visual sensors, the number of sensors may be two or more, and the multiple sensors may be arranged left and right, up and down, or in any other manner.

According to the relative pose of the vehicle and at least one scene element, the position, angle and other information of the visual sensor installed on the vehicle relative to the surrounding scene elements can be determined, and then the scene image observed by the visual sensor can be determined.

Optionally, the number of the multiple scene images may be consistent with the number of visual sensors. For example, if the control system uses the images collected by the left and right visual sensors of the vehicle as input, the multiple scene images generated in this step may be It includes the scene image observable by the left vision sensor and the scene image observable by the right vision sensor.

Step 204: Output the multiple scene images, so that the control system generates corresponding control signals according to the multiple scene images.

For example, the simulator can output corresponding multiple scene images according to the driving control signals obtained from the driving control system, and the output multiple scene images can be used to simulate images with parallax captured by multiple visual sensors of the vehicle. Specifically, the scene image may be a scene image that can be observed after the vehicle moves in the scene model in response to the control signal. The simulator can output a video stream with parallax as the vehicle continues to move forward. This creates a process of constantly controlling and updating the scene image, enabling testing of the driving control system.

Optionally, the scene image may not be directly output to the control system by the emulator, but input to the control system by other devices, which is not limited in this embodiment.

The solution of this embodiment has been described above by taking the test of the driving control system of the vehicle as an example. On this basis, the vehicle can also be replaced with other movable platforms, and the driving control system can be replaced with other control systems, so as to realize the test of other control systems based on stereo vision.

In an example, the movable platform may be a ship, and the scene model may be a model corresponding to an actual working scene of the ship, for example, a model corresponding to a river. The ship may be installed with a control system and a plurality of visual sensors, and the method described in this embodiment can implement the test of the control system of the ship, thereby assisting in the realization of the automatic driving test verification of the ship.

In another example, the movable platform may be an intelligent robot, and the scene model may be a model corresponding to an actual working scene of the robot, for example, a model corresponding to a shopping mall or a warehouse. The intelligent robot can be equipped with a control system and a plurality of visual sensors, and the method described in this embodiment can realize the test of the control system of the intelligent robot, thereby assisting in realizing the autonomous movement test verification of the intelligent robot.

The simulation test method provided in this embodiment can be applied to a simulation test system. The simulation test system is used to test a control system of a movable platform based on a virtual scene model. The scene model includes a plurality of scene elements, and the control system for outputting a control signal for controlling the movable platform based on scene images observed by at least two visual sensors, wherein the simulation test system can specifically acquire the control signal of the movable platform output by the control system , based on the control signal, simulate the movement of the movable platform in the scene model, obtain the relative pose between the movable platform and the scene element, and generate a plurality of scene images according to the relative pose , the plurality of scene images include scene images observed by at least two of the visual sensors when the movable platform moves in the scene model, and output the plurality of scene images, so that the control system Corresponding control signals are generated according to the plurality of scene images, so that the actual image input of the control system can be simulated by generating scene images observable by at least two vision sensors, and the simulation test of the stereo vision-based control system can be realized. Efficiently test and verify the basic functions of the control system, solve practical problems such as unreliability, incomplete scene test categories and high investment caused by relying on a large number of road tests, effectively improve test efficiency and reduce test costs.

On the basis of the technical solutions provided in the foregoing embodiments, optionally, generating a plurality of scene images according to the relative poses may include:

According to the relative pose between the movable platform and the scene element, an imaging change of the scene element observable by the movable platform is determined, and a plurality of scene images are generated based on the imaging change.

The relative pose may include position and/or angle information of the movable platform relative to one or more scene elements in the scene model. Based on the relative pose, scene elements that can be observed by the movable platform can be determined. The imaging information of the scene element on the vision sensor mounted on the movable platform may be determined based on the relative pose.

When the relative pose changes, the imaging of the scene element will also change. Based on this imaging change, the imaging information of the scene element observed by the vision sensor after the relative pose is changed can be determined, so as to generate a corresponding image. scene image.

The imaging changes of the scene elements that can be observed by the movable platform are determined by the relative pose, and multiple scene images are generated based on the imaging changes, so that the scene images that can be observed by the movable platform can be accurately constructed, and the simulation is improved. Efficiency and accuracy of testing.

In an optional implementation manner, when generating a plurality of scene images according to the relative pose, all the scene images may be generated specifically according to the relative pose and stereo vision parameters between the movable platform and the scene element. describe multiple scene images.

Wherein, the stereo vision parameters may include, but are not limited to, the height of the vision sensor, the shooting angle, the baseline parameters, and the like.

According to the relative pose and stereo vision parameters of the movable platform and at least one scene element, information such as the position and angle of the vision sensor installed on the movable platform relative to the surrounding scene elements can be determined, and then the vision sensor can be determined. The observed scene image, so that the scene image that can be observed by each vision sensor can be generated more accurately, and the accuracy of the simulation test can be further improved.

In an optional implementation manner, the stereo vision parameter is determined by the installation pose of the at least two vision sensors and/or the relative pose between the at least two vision sensors.

In an optional implementation manner, outputting the plurality of scene images includes:

Sending the plurality of scene images to the control system through an image output interface.

In an optional implementation manner, sending the plurality of scene images to the control system through an image output interface includes:

converting the plurality of scene images into a plurality of scene images in a preset format;

Sending the plurality of scene images in the preset format to the control system through an image output interface.

In an optional implementation manner, outputting the multiple scene images, so that the control system generates corresponding control signals according to the multiple scene images, includes:

Send the plurality of scene images to a display device to display the plurality of scene images through the display device, so that the control system outputs corresponding images based on the at least two visual sensors photographing the display device. control signal;

The number of the visual sensors is the same as the number of displayed scene images; the at least two visual sensors are in one-to-one correspondence with the plurality of scene images, and the visual sensors are used to photograph the corresponding scene images.

In an optional implementation manner, the number of the display devices is the same as the number of the visual sensors, at least two of the display devices are in one-to-one correspondence with the at least two visual sensors, and each visual sensor is used for Shoot the picture displayed by the corresponding display device;

Correspondingly, sending the plurality of scene images to a display device, so as to display the plurality of scene images through the display device, includes:

Each scene image is sent to its corresponding display device for display, so as to display the plurality of scene images through at least two of the display devices.

In an optional implementation manner, before each scene image is sent to its corresponding display device for display, the method further includes:

Image conversion is performed on at least part of the multiple scene images according to the calibration parameters corresponding to the visual sensor.

In an optional implementation manner, multiple scene images are generated according to the relative pose, including:

A plurality of scene images are generated according to the relative poses between the movable platform and the scene elements, stereo vision parameters, and calibration parameters corresponding to the vision sensor.

In an optional implementation, the method further includes:

The vision sensor is calibrated by a camera calibration method to determine the calibration parameters of the vision sensor.

In an optional implementation manner, the display device is a 3D display device;

Sending the plurality of scene images to a display device to display the plurality of scene images through the display device, including:

Sending the plurality of scene images to the 3D display device, so that the 3D display device displays the plurality of scene images through 3D projection.

In an optional implementation, the method further includes:

Determine corresponding sensing information according to the relative pose between the movable platform and the scene element;

The sensing information is output, so that the control system determines a corresponding control signal according to the plurality of scene images and the sensing information.

In an optional implementation manner, the sensing information includes point cloud data corresponding to the scene element.

In an optional implementation, the method further includes:

determining the operating state of the movable platform according to the control signal;

The control system is evaluated according to the operating state of the movable platform.

In an optional implementation manner, the movable platform is a vehicle, and the control system is a driving control system applied to the vehicle.

The technical solution of the present application will be described in detail below with some specific embodiments. The various embodiments given in this application may be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. For content that is not described in detail in some embodiments, reference may be made to related descriptions in other embodiments.

FIG. 3A is a schematic flowchart of another simulation testing method provided by an embodiment of the present application. The method in this embodiment may be executed by an emulator, or may be implemented jointly by the emulator and other devices. As shown in Figure 3A, the method may include:

Step 301: Acquire a control signal output by a control system to be tested, where the control signal is a control signal output by the control system according to historically input scene images and a preset control model for controlling the movable platform.

Step 302 , output at least two scene images according to the control signal and the virtual scene model.

Wherein, the scene model includes multiple scene elements.

The at least two scene images include a first scene image and a second scene image, the first scene image includes a first pixel area, the second scene image includes a second pixel area, the first pixel area and the The second pixel area describes the same scene element; the position of the first pixel area in the first scene image is between the position of the second pixel area in the second scene image The deviation is determined according to the relative pose between the visual sensors of the movable platform.

In this embodiment, the explanation of related concepts such as the control system, the simulator, the scene model, and the scene elements, as well as the specific implementation principles and processes of each step, can refer to the foregoing embodiments, and details are not repeated here.

In this embodiment, the outputted multiple scene images may be multiple scene images with parallax. FIG. 3B is a schematic diagram of positions of scene elements in a scene image according to an embodiment of the present application. As shown in FIG. 3B , the left and right images are two output scene images, and the two scene images have parallax.

Specifically, for any scene element such as a tree, in different scene images, the position of the tree can be different, because the two eyes of a person have a certain distance, and the left and right eyes look at the object. Seeing from different angles will cause objects to be in different positions in the view of the left and right eyes, resulting in parallax. The control system based on stereo vision uses this principle to construct the depth information of the surrounding environment through multiple images with parallax, so as to control the vehicle more accurately.

When testing the driving control system, at least two scene images generated according to the scene model may be output to the control system, and the same scene element in the scene model may be located in different pixel areas in different scene images.

Referring to FIG. 3B , the position of the tree in the first scene image may be different from its position in the second scene image. Specifically, the tree may be located in the first pixel area in the first scene image, and in the second scene image may be located in the second pixel area. The position of the first pixel area in the first scene image and the position of the second pixel area in the second scene image are not completely coincident, but have a certain deviation.

In the scenario where the driving control system of the vehicle is simulated and tested, between the position of the first pixel area in the first scene image and the position of the second pixel area in the second scene image The positional deviation can be determined according to the relative pose between the visual sensors of the vehicle.

The closer the positions and shooting angles of the at least two visual sensors of the vehicle are, the smaller the positional deviation of the same scene element in different scene images; the greater the difference between the positions and shooting angles of the at least two visual sensors of the vehicle, the same The positional deviation of scene elements in different scene images is also greater.

The simulation testing method provided in this embodiment can acquire the control signal output by the control system to be tested, where the control signal is output by the control system according to the historically input scene image and the preset control model for controlling the movable platform and output at least two scene images according to the control signal and the virtual scene model, wherein the scene model includes a plurality of scene elements, and the scene images are the movable platform in response to the control The scene images of the scene elements observed after the signal moves, the positional deviations of the scene elements in different scene images are determined according to the relative poses between the visual sensors of the movable platform, so that multiple scenes with parallax can be passed through. The image simulates the actual image input of the control system, realizes the simulation test of the control system based on stereo vision, tests and verifies the basic functions of the control system economically and efficiently, and solves the unreliability and incomplete test categories caused by relying on a large number of road tests. And practical problems such as high investment, effectively improve the test efficiency and reduce the test cost.

FIG. 4 is a schematic flowchart of another simulation testing method provided by an embodiment of the present application. In this embodiment, on the basis of the technical solutions provided by the above-mentioned embodiments, at least two scene images are displayed through a display device. As shown in Figure 4, the method may include:

Step 401: Obtain a control signal output by a control system to be tested, where the control signal is a control signal output by the control system according to historically input scene images and a preset control model for controlling the movable platform.

For the specific implementation principle and process of step 401 in this embodiment, reference may be made to the foregoing embodiments, and details are not repeated here.

Step 402: Determine at least two scene images according to the control signal and the virtual scene model.

In this embodiment, at least two scene images can be output according to the control signal and the virtual scene model through steps 402 to 403 . Specifically, at least two scene images may be determined according to the control signal and the virtual scene model, and then the at least two scene images are output.

Optionally, outputting at least two scene images according to the control signal and the virtual scene model may include: determining at least two scene images according to the control signal, the virtual scene model, and stereoscopic parameters; outputting the At least two scene images.

Wherein, the stereo vision parameter may be any parameter used to impart parallax to the scene image. Optionally, the stereo vision parameter is determined by the installation pose of the vision sensor of the movable platform and/or the relative pose between the vision sensors of the movable platform.

Optionally, the installation posture may include an installation position and/or an installation angle, etc., and the relative posture may include a relative position and/or angle of the visual sensors to each other, and the like. By determining the stereo vision parameters by the installation pose and relative pose of the vision sensor, the parallax between the scene images can be determined more accurately, and the accuracy of the simulation test can be improved.

Specifically, the stereo vision parameters may include baseline (Baseline) parameters between vision sensors, and the like. The baseline parameter is used to represent the center distance of the visual sensor, and the parallax size between the output scene images is set by the baseline parameter, so that the scene image can be generated quickly and accurately.

Optionally, determining at least two scene images according to the control signal, the virtual scene model and the stereo vision parameters may include: determining the movable platform and the scene according to the control signal and the virtual scene model. relative pose of the element; at least two scene images are determined according to the relative pose of the movable platform and the scene element and stereo vision parameters.

For example, the movable platform is a vehicle. After the driving control system outputs a driving control signal, the vehicle is regarded as moving in the virtual scene model, which is equivalent to a change in the position of the vehicle relative to scene elements such as trees. At least two scene images to be output can be determined according to the relative pose and stereo vision parameters of the vehicle and the scene elements.

Wherein, the specific method for determining the scene image according to the relative pose and the stereo vision parameter can be implemented through a simulation experiment, which is not repeated in this embodiment of the present application. The relative pose of the movable platform and the scene elements is determined by the control signal and the virtual scene model, and at least two scene images are determined according to the relative pose and stereo vision parameters, which can accurately simulate the observation of the movable platform in the actual scene. images to further improve the test accuracy.

Step 403: Send the at least two scene images to a display device, so as to display the at least two scene images through the display device, so that the control system is based on the images captured by the at least two visual sensors and the preset The control model determines the corresponding control signal.

In this embodiment, at least two scene images can be output through step 403 .

The visual sensor may be a device capable of capturing images, such as a camera, a camera, or the like. The number of the visual sensors is the same as the number of output scene images; the at least two visual sensors are in one-to-one correspondence with the at least two scene images, and the visual sensors are used to photograph the corresponding scene images; the The relative pose between the at least two vision sensors is determined by the relative pose between the vision sensors of the movable platform.

Specifically, the at least two visual sensors are in a one-to-one correspondence with the at least two scene images, which may mean that there are n visual sensors and n scene images, wherein the ith visual sensor corresponds to the ith scene image , that is, the ith visual sensor is used to capture the ith scene image displayed by the display device, where i takes a value from 1 to n, and n is a positive integer ≥ 2.

Optionally, the at least two visual sensors used in the simulation test can be used to simulate the visual sensors used by the vehicle in practical applications. The relative pose between the at least two vision sensors may be determined from the relative pose between the visual sensors in the practical application. Since the at least two vision sensors are used to simulate the vision sensors of the actual vehicle, the at least two vision sensors used in the test may be configured according to the vision sensors of the actual vehicle, for example, may be configured according to the vision sensors of the actual vehicle The number, resolution, position, shooting angle, etc. of the sensors are used to determine the number, resolution, position, shooting angle, etc. of the at least two visual sensors during the test, so that the visual sensor during testing is kept with the visual sensor during actual use. Consistent, improving the accuracy of the test.

The simulation test method provided in this embodiment can realize the test of the control system and the visual sensor. The simulator can play the scene image to the visual sensor for "viewing" through the display device, and the control system processes the image collected by the visual sensor and outputs the corresponding control signal. This test method can realize the hardware-in-the-loop experiment and completely test the The hardware and software parts of the whole system fill the experimental gap of the stereo vision hardware-in-the-loop simulation test.

On the basis of the technical solutions provided in the foregoing embodiments, optionally, the number of the display devices may be one or more, for example, multiple display devices may be set to display corresponding scene images respectively, or, It is also possible to set a display device to display each scene image on different display areas of the display device.

Optionally, the number of the display devices is the same as the number of the visual sensors, the at least two display devices are in one-to-one correspondence with the at least two visual sensors, and each visual sensor is used to display the corresponding display device. screen to shoot. Correspondingly, sending the at least two scene images to a display device to display the at least two scene images through the display device may include: sending each scene image to its corresponding display device for display, so as to display the at least two scene images through the display device. The at least two scene images are displayed by at least two of the display devices.

Wherein, the at least two display devices are in one-to-one correspondence with the at least two visual sensors, which may mean that there are n visual sensors and n display devices, wherein the ith visual sensor corresponds to the ith display device, That is, the i-th visual sensor is used to capture a picture of the i-th display device, where i takes a value from 1 to n, and n is a positive integer ≥ 2.

FIG. 5 is a schematic diagram of a test architecture based on multiple display devices according to an embodiment of the present application. As shown in Figure 5, the control system and the visual sensor connected to it constitute the tested system. There are two visual sensors, which are marked as the left camera and the right camera respectively. There can also be two display devices, which are marked as the left display device respectively. and the display device on the right, correspondingly, there are also two output scene images, which are respectively recorded as the left image and the right image.

During the simulation test, after the emulator generates the left image and the right image with parallax, the left image can be sent to the left display device, and the right image can be sent to the right display device, and the two display devices can be synchronized respectively. Play the parallax images on the left and right sides. The left camera and the right camera shoot the pictures displayed by the left display device and the right display device respectively, so that different images can be collected by different cameras and input to the control system for processing, realizing the combination of software and hardware. test.

It should be noted that the monocular system can perceive the change of the environment through the change of the frame before and after, while the image obtained by the binocular system has parallax. Through the two binocular images at a single moment, the depth information of the target object can be inferred. , this embodiment is based on this principle, using two display devices to play the video stream formed by the scene image respectively, and using two cameras to capture the video stream played by each display device and transmit it to the control system, realizing the simulation environment. The stereo imaging experiment, so that the stereo simulation video stream can be generated to the tested system under indoor conditions to achieve the same test effect as the real scene.

By setting up multiple display devices, and the display devices, visual sensors and scene images all have a corresponding relationship, a scene image is displayed by the corresponding display device and photographed by the corresponding visual sensor, so that each scene image does not interfere with each other, improving the the accuracy of the image output.

Optionally, the vision sensor may also be calibrated by a camera calibration method to determine the calibration parameters of the vision sensor, and determine the output scene image according to the calibration parameters. The camera calibration method may include calibration methods such as the checkerboard method.

FIG. 6 is a schematic diagram of a calibration pattern provided by an embodiment of the present application. When calibrating vision sensors, the pattern shown in Figure 6 can be used. Specifically, the two display devices can play the calibration pattern respectively, collect the calibration image by the corresponding vision sensor, use the calibration algorithm to calculate the rotation and/or translation parameters of the pattern, mark it as the calibration parameter of the vision sensor, and feed the parameter back The simulator is provided with reverse rotation and/or translation of the output image by the simulator, so that the images collected by the two vision sensors can be aligned, so as to achieve good depth matching of the stereo vision algorithm.

For example, after calibrating the vision sensor, it is determined that the left image needs to be rotated 5° clockwise relative to the right image, then in the subsequent testing process, each image displayed by the left display device can be rotated 5° clockwise to ensure Left and right images are aligned.

In one example, before each scene image is sent to its corresponding display device for display, image transformation may be performed on at least part of the at least two scene images according to calibration parameters corresponding to the visual sensor.

Specifically, the simulator may first obtain at least two scene images with parallax, and then perform image transformation on at least part of the at least two scene images according to the calibration parameters, for example, perform rotation and/or translation, and finally Get aligned images.

Optionally, the image transformation can be implemented through an emulator, or can be implemented by adding other modules after the emulator.

When the calibration parameters change, the at least two scene images can be directly adjusted according to the changed calibration parameters, and the solution is flexible and easy to implement.

In another example, outputting at least two scene images according to the control signal and the virtual scene model may include: according to the control signal, the virtual scene model, stereo vision parameters, and calibration parameters corresponding to the vision sensor, determining at least two scene images; outputting the at least two scene images.

Specifically, the simulator can directly obtain at least two scene images with parallax and aligned according to the control signal, the virtual scene model, the stereo vision parameters and the calibration parameters corresponding to the vision sensor, so that the final output scene image can be quickly determined , to improve the efficiency of simulation testing.

FIG. 7 is a schematic diagram of a test architecture based on an optical system provided by an embodiment of the present application. The solution shown in FIG. 7 is based on the solution shown in FIG. 5 , and an optical system is added. The optical system may be disposed between the display device and the visual sensor, and the optical system is used to perform optical conversion on the image output by the display device, so that the converted image matches the field of view of the visual sensor angle (FOV).

Wherein, the number of the optical systems may be one or more, one optical system may be configured for each display device, or one optical system may be shared by multiple display devices. The structure of the optical system can be designed according to actual needs, as long as the image can be converted to match the field of view of the vision sensor.

Since the field of view of the vehicle vision sensor is usually relatively large, in order to match the field of view of the visual sensor, the direct display method through the display device usually makes the entire test system very large. Therefore, an optical system can be introduced in this embodiment, and the optical system can perform optical conversion on the image output by the display device and enlarge the output image, thereby matching the field of view of the vision sensor, which can effectively reduce the volume of the imaging part without using a huge display equipment, reducing the test footprint.

FIG. 8 is a schematic diagram of a test architecture based on a 3D display device provided by an embodiment of the present application. In the architecture, the display device may be a 3D display device. Correspondingly, sending the at least two scene images to a display device to display the at least two scene images through the display device may include: sending the at least two scene images to the 3D display device, so that the 3D display device displays the at least two scene images by means of 3D projection.

Among various types of light, there is no fixed polarization direction, that is, the intensity of light waves vibrating in all directions is the same, which is usually called natural light. For example, sunlight is one of the most common natural light; there is a fixed polarization direction. , often referred to as polarized light, the light emitted by the screen of a display device can be polarized light.

Optionally, different polarization directions can be set for different scene images, thereby forming 3D images. The vision sensor can be provided with a polarizer that is consistent with the polarization direction of the polarized light emitted by the display device. The polarizer only allows light with a specific polarization direction to enter the vision sensor, so that images with a specific polarization direction can be accurately collected without being affected by other polarizations. The effect of the orientation of the image.

Optionally, the polarization directions of the two scene images may be orthogonal. Each vision sensor is aligned with the polarization direction of the corresponding image, respectively.

For example, the polarization direction of the first scene image is direction 1, the polarization direction of the second scene image is direction 2, the first vision sensor is used to collect the first scene image, and the second vision sensor is used to collect the second scene image, then the first The polarization direction of one vision sensor may be direction 1, and the polarization direction of the second vision sensor may be direction 2.

As shown in Figure 8, the control system and the visual sensor connected to it constitute the tested system. There are two visual sensors, which are marked as the left camera and the right camera respectively, and there can be only one display device, which is a 3D display device. The device is capable of displaying 3D images, so that two scene images can be displayed by one 3D display device for the left camera and the right camera to capture respectively.

In this implementation scheme, through 3D projection, at least two scene images can be played in 3D technology, and the two cameras to be tested can be equipped with orthogonal polarizers, similar to watching 3D movies, so that the two cameras can be Each camera can watch a picture at the same time, but due to the different polarization directions, the images collected by the two cameras are different, and the back-end stereo restoration of the control system can also be realized.

By sending at least two scene images to the 3D display device, so that the 3D device displays the at least two scene images in a 3D projection manner, multiple scene images can be displayed on the same display device, and each scene image They do not interfere with each other, thereby reducing the number of equipment, reducing the area of the test site and reducing the test cost on the basis of ensuring the test accuracy.

FIG. 9 is a schematic diagram of a test architecture based on an image output interface provided by an embodiment of the present application. In the solution shown in FIG. 9 , outputting at least two scene images may include: sending the at least two scene images to the control system through an image output interface.

Wherein, the image output interface may be integrated with the simulator, or may be provided separately from the simulator, or the image output interface may also be integrated with the control system.

As shown in FIG. 9 , after acquiring the control signal output by the control system, the simulator can determine at least two scene images according to the control signal and the virtual scene model, and then send the at least two scene images through the image output interface Send it to the control system, and the control system can continue to generate and output the corresponding control signal according to the scene image.

In this implementation, the actual imaging input of the vision sensor can be discarded, and at least two scene images can be input to the control system in the form of digital signals directly through the image output interface. This method belongs to the pseudo hardware-in-the-loop implementation. The optical imaging part of the vision sensor has not been tested, but the control system can be tested, thereby realizing the testing of the system software.

Through the above implementation manner, the control system can be tested independently, so as to further reduce the equipment used in the simulation test and improve the flexibility of the simulation test. Generally speaking, the control system can be tested separately in the early stage. After the control system has passed the test, a visual sensor can be added to further test the software and hardware. In this way, two tests can be implemented through a single simulator, which improves the test efficiency.

Optionally, sending the at least two scene images to the control system through an image output interface may include: converting the at least two scene images into at least two scene images in a preset format; The formatted at least two scene images are sent to the control system through an image output interface.

Wherein, the preset format may be the format output by the vision sensor in actual use. For example, in actual use, the output format of the vision sensor to the control system is USB format. Then, during the test, the emulator can convert the scene image in HDMI format determined according to the control signal and the virtual scene model into USB format and send it to the control system through the image output interface, so that the test environment is closer to the actual use scene. Meet the input requirements of the control system, so that the test can be carried out smoothly.

FIG. 10 is a schematic diagram of a test architecture based on a direct connection of an emulator provided by an embodiment of the present application. In the solution shown in FIG. 10 , outputting at least two scene images may include: directly sending the at least two scene images to the control system.

As shown in FIG. 10 , the simulator can be directly connected with the control system, and the generated at least two scene images are directly sent to the control system.

For the control system, whether it is connected to a visual sensor or directly to an emulator, the interface and protocol can be the same, and the control system does not need to pay attention to whether the acquired image is a real captured image or directly received image data. In the implementation shown in FIG. 10 , the image output by the emulator can be directly sent to the control system without passing through other modules. The structure is simple and easy to implement, which further reduces the cost of testing.

The above embodiment provides multiple implementation methods for stereo simulation testing. It should be noted that more implementation methods can be extended on this basis, as long as it is ensured that the image obtained by the control system is an image with parallax.

Optionally, the scene image determined by the simulator is a single scene image, and an image output interface can be set after the simulator, and the image output interface can output the at least two scene images according to the single scene image, thereby effectively Reduce the burden on the emulator.

Wherein, the single scene image may be an observable scene image at any position of the movable platform, for example, a scene image observable by one of the vision sensors, or observable at a midpoint between two vision sensors to the scene image.

The image output interface may store stereo vision parameters to convert a single scene image into multiple scene images with parallax. In addition, the image output interface may also store the information of each scene element in the virtual scene model, so as to restore the scene image observable by each visual sensor more accurately.

Optionally, a module can also be added behind the vision sensor to add parallax to the scene image. For example, if the scene images received by at least two visual sensors are the same, a conversion module is added after the visual sensors, and the scene images are converted by the conversion module to obtain at least two scene images with parallax and output them to Control System.

To sum up, in the embodiment of the present application, at least two scene images input to the control system have parallax, but the specific link in which the parallax is obtained is not limited in the embodiment of the present application.

Based on the technical solutions provided by the above embodiments, optionally, the method further includes: determining corresponding sensing information according to the control signal and the virtual scene model; outputting the sensing information to The control system is made to determine a corresponding control signal according to the at least two scene images and the sensing information.

In actual use, various types of sensors can be provided on the movable platform, and are not limited to vision sensors. Correspondingly, during the simulation test, by designing a variety of sensor information to input into the control system, the operation of the control system in different states can be effectively tested to meet the needs of different application scenarios.

The sensing information may be any type of sensing information, including but not limited to: wind speed, temperature, humidity, weather, and the like.

Optionally, the sensing information may include point cloud data. In actual use, point cloud data can be detected by lidar, and lidar can be used to assist in perceiving the surrounding environment.

Correspondingly, during the simulation test, determining the corresponding sensing information according to the control signal and the virtual scene model may include: determining the movable platform and the scene according to the control signal and the virtual scene model. relative poses between elements; point cloud data is determined according to the relative poses between the movable platform and the scene elements.

Specifically, during the simulation test, in addition to outputting at least two scene images with parallax, the simulator can also output point cloud data corresponding to the scene elements in the virtual scene model. The point cloud data can describe the depth information of scene elements, so that the control system can determine the corresponding control signal according to the at least two scene images and the point cloud data, which can not only test the response of the control system to the scene images, but also The response of the test control system to the point cloud data can effectively improve the dimension of the test and improve the test effect.

On the basis of the technical solutions provided in the foregoing embodiments, optionally, the method further includes: determining the operating state of the movable platform according to the control signal; and, according to the operating state of the movable platform, The control system was evaluated.

Taking the control system as a driving control system applied to a vehicle and the movable platform as a vehicle as an example, the simulator can determine at least two scene images or sensor information to be displayed through the control signal and the virtual scene model, At least two scene images or sensing information are simulations of real environment data, and the driving control system can identify at least two scene images or sensing information, and obtain corresponding driving control strategies according to the identified information and preset driving control strategies. Decisions are control signals. For example, if a zebra crossing is detected, a deceleration signal is output.

After the driving control system outputs a control signal, the simulator will determine, according to the control signal, the running state of the vehicle driving according to the control signal, such as the position of the vehicle in the lane, the distance to surrounding obstacles, and so on.

Then, the control system can be evaluated according to the running state of the vehicle, for example, whether to deviate from the driving lane, whether to run a red light, whether it is too close to an obstacle, etc., and output the evaluation of the driving control system according to the judgment result. . The evaluation can be a rating or whether it is qualified or not.

For example, if the driving control system controls the vehicle to run a red light, collide with an obstacle or other behaviors or dangerous behaviors that do not follow the traffic rules during the simulation test, it can be considered that the driving control system is not. If qualified, the algorithm needs to be re-optimized.

If the driving control system runs smoothly and smoothly throughout the simulation test process, the control system can be considered qualified. After passing the simulation test, an actual road test or other tests can be arranged, and after all the tests are completed, the driving control system can be put into use.

Through the running state of the vehicle, the evaluation of the control system can be effectively realized, the requirements of the simulation test can be met, most of the test problems can be solved or converged in advance, the cost of the later road test can be reduced, and the test efficiency can be improved.

FIG. 11A is a schematic structural diagram of a simulation testing apparatus provided by an embodiment of the present application. The device is applied to a simulation test system; the simulation test system is used to test a control system of a movable platform based on a virtual scene model, and the scene model includes a plurality of scene elements; the control system is used to, based on at least two The scene image observed by the vision sensor outputs a control signal for controlling the movable platform. As shown in Figure 11A, the apparatus may include:

an acquisition module 1101, configured to acquire the control signal of the movable platform output by the control system;

A simulation module 1102, configured to simulate the movement of the movable platform in the scene model based on the control signal, to obtain the relative pose between the movable platform and the scene element;

A generating module 1103, configured to generate a plurality of scene images according to the relative poses, where the plurality of scene images include scenes observed by at least two of the visual sensors when the movable platform moves in the scene model image;

The output module 1104 is configured to output the plurality of scene images, so that the control system generates corresponding control signals according to the plurality of scene images.

In a possible implementation manner, when the generating module 1103 generates multiple scene images according to the relative pose, it is specifically used for:

The plurality of scene images are generated according to the relative pose and stereo vision parameters between the movable platform and the scene elements.

In a possible implementation manner, the stereo vision parameter is determined by the installation pose of the at least two vision sensors and/or the relative pose between the at least two vision sensors.

In a possible implementation manner, when the output module 1104 outputs the multiple scene images, it is specifically used for:

In a possible implementation manner, when the output module 1104 sends the plurality of scene images to the control system through an image output interface, it is specifically configured to:

Sending the multiple scene images in the preset format to the control system through an image output interface.

In a possible implementation manner, the output module 1104 is specifically used for:

In a possible implementation manner, the number of the display devices is the same as the number of the visual sensors, at least two of the display devices are in one-to-one correspondence with the at least two visual sensors, and each visual sensor is used for Shoot the picture displayed by the corresponding display device;

Correspondingly, when the output module 1104 sends the plurality of scene images to a display device to display the plurality of scene images through the display device, the output module 1104 is specifically configured to:

In a possible implementation manner, before sending each scene image to its corresponding display device for display, the output module 1104 is further configured to:

In a possible implementation manner, the generating module 1103 is further configured to:

In a possible implementation manner, the display device is a 3D display device;

When the output module 1104 sends the plurality of scene images to a display device to display the plurality of scene images through the display device, the output module 1104 is specifically configured to:

In a possible implementation manner, the output module 1104 is further configured to:

In a possible implementation manner, the sensing information includes point cloud data corresponding to the scene element.

In a possible implementation manner, the movable platform is a vehicle, and the control system is a driving control system applied to the vehicle.

The simulation testing apparatus provided in this embodiment can be used to execute the simulation testing method shown in FIG. 2 , and the specific implementation principles and effects thereof can refer to the foregoing embodiments, which will not be repeated here.

FIG. 11B is a schematic structural diagram of another simulation testing apparatus provided by an embodiment of the present application. As shown in Figure 11B, the apparatus may include:

The acquisition module 1111 is used to acquire the control signal output by the control system to be tested, the control signal is the control signal output by the control system according to the historically input scene image and the preset control model for controlling the movable platform;

an output module 1112, configured to output at least two scene images according to the control signal and the virtual scene model;

Wherein, the virtual scene model includes a plurality of scene elements;

In a possible implementation manner, the output module 1112 is specifically used for:

determining at least two scene images according to the control signal, the virtual scene model and the stereo vision parameters;

The at least two scene images are output.

According to the control signal and the virtual scene model, determine the relative pose of the movable platform and the scene element;

At least two scene images are determined according to the relative poses of the movable platform and the scene elements and stereo vision parameters.

In a possible implementation manner, the stereo vision parameter is determined by the installation pose of the vision sensor of the movable platform and/or the relative pose between the vision sensors of the movable platform.

In a possible implementation manner, when the output module 1112 outputs at least two scene images, it is specifically used for:

The at least two scene images are sent to the control system through an image output interface.

In a possible implementation manner, when the output module 1112 sends the at least two scene images to the control system through an image output interface, it is specifically used for:

converting the at least two scene images into at least two scene images in a preset format;

The at least two scene images in the preset format are sent to the control system through an image output interface.

Send the at least two scene images to a display device, so that the at least two scene images are displayed by the display device, so that the control system determines according to the images captured by the at least two visual sensors and a preset control model Corresponding control signal;

The number of the visual sensors is the same as the number of output scene images; the at least two visual sensors are in one-to-one correspondence with the at least two scene images, and the visual sensors are used to photograph the corresponding scene images.

Correspondingly, when the output module 1112 sends the at least two scene images to a display device, so as to display the at least two scene images through the display device, the output module 1112 is specifically configured to:

Each scene image is sent to its corresponding display device for display, so as to display the at least two scene images through at least two of the display devices.

In a possible implementation manner, the output module 1112 is further configured to:

Before sending the scene image to the corresponding display device for display, image conversion is performed on at least part of the at least two scene images according to the calibration parameters corresponding to the visual sensor.

In a possible implementation manner, when outputting at least two scene images according to the control signal and the virtual scene model, the output module 1112 is specifically configured to:

Determine at least two scene images according to the control signal, the virtual scene model, the stereo vision parameters, and the calibration parameters corresponding to the vision sensor;

The at least two scene images are output.

In a possible implementation manner, the display device is a 3D display device;

When the output module 1112 sends the at least two scene images to a display device, so as to display the at least two scene images through the display device, the output module 1112 is specifically configured to:

The at least two scene images are sent to the 3D display device, so that the 3D display device displays the at least two scene images by means of 3D projection.

Determine corresponding sensing information according to the control signal and the virtual scene model;

The sensing information is output, so that the control system determines a corresponding control signal according to the at least two scene images and the sensing information.

In a possible implementation manner, the sensing information includes point cloud data corresponding to the scene element;

When determining the corresponding sensing information according to the control signal and the virtual scene model, the output module 1112 is specifically used for:

determining the relative pose between the movable platform and the scene element according to the control signal and the virtual scene model;

Point cloud data is determined according to the relative pose between the movable platform and the scene element.

In a possible implementation manner, the control system is a driving control system applied to a vehicle, and the movable platform is a vehicle.

The simulation testing apparatus provided in this embodiment can be used to execute the simulation testing methods in the embodiments shown in FIG. 3A to FIG. 10 , and the specific implementation principles and effects can be found in the foregoing embodiments, which are not repeated here.

FIG. 12A is a schematic structural diagram of a simulation testing system provided by an embodiment of the present application. The simulation test system is used for testing the control system of the movable platform based on a virtual scene model, the scene model includes a plurality of scene elements; the control system is used for, based on the scene images observed by at least two visual sensors, output Control signals for controlling the movable platform.

As shown in FIG. 12A, the system may include: an emulator 1201 and an image output device 1202;

The simulator 1201 is configured to obtain the control signal of the movable platform output by the control system, and based on the control signal, simulate the movement of the movable platform in the scene model, and obtain the difference between the movable platform and the movable platform. relative poses between the scene elements, and generate an observable scene image based on the relative poses;

The image output device 1202 is configured to acquire the scene image generated by the simulator 1201, and output a plurality of scene images according to the acquired scene image, and the plurality of scene images include when the movable platform moves in the scene model. at least two scene images observed by the visual sensors, so that the control system generates corresponding control signals according to the plurality of scene images.

In a possible implementation manner, when the simulator 1201 generates an observable scene image based on the relative pose, it is specifically used for:

The plurality of scene images are generated based on relative poses between the movable platform and the scene elements.

In a possible implementation manner, when the simulator 1201 generates multiple scene images according to the relative pose, it is specifically used for:

In a possible implementation manner, the image output device 1202 includes an image output interface, and the emulator 1201 is further used for:

Sending the plurality of scene images to the image output interface, so that the image output interface sends the plurality of scene images to the control system.

In a possible implementation manner, when the simulator 1201 sends the multiple scene images to the image output interface, it is specifically used for:

Sending the plurality of scene images in the preset format to the image output interface.

In a possible implementation manner, the image output device 1202 includes a display device for displaying the plurality of scene images;

The simulator 1201 is further configured to: send the plurality of scene images to a display device, so as to display the plurality of scene images through the display device, so that the control system captures the display according to at least two visual sensors The control signal corresponding to the image output obtained by the device;

Correspondingly, when the simulator 1201 sends the plurality of scene images to a display device to display the plurality of scene images through the display device, it is specifically used for:

In a possible implementation manner, the emulator 1201 is also used for:

Before sending the scene image to the corresponding display device for display, image conversion is performed on at least part of the plurality of scene images according to the calibration parameter corresponding to the visual sensor.

In a possible implementation manner, the emulator 1201 is also used for:

In a possible implementation manner, the display device is a 3D display device, and the 3D display device displays the plurality of scene images in a 3D projection manner.

In a possible implementation manner, the system further includes: an optical system;

The optical system is arranged between the display device and the visual sensor, and the optical system is used to perform optical conversion on the scene image output by the display device, so that the converted scene image matches the visual field of the visual sensor. field angle.

In a possible implementation manner, the scene image generated by the simulator 1201 is a single scene image, and the image output device 1202 is specifically configured to output the multiple scene images according to the single scene image.

In a possible implementation manner, the emulator 1201 is also used for:

The simulation testing system provided in this embodiment can be used to execute the simulation testing method shown in FIG. 2 , and the specific implementation principles and effects thereof can refer to the foregoing embodiments, which will not be repeated here.

FIG. 12B is a schematic structural diagram of another simulation testing system provided by an embodiment of the present application. As shown in FIG. 12B , the system may include: an emulator 1211 and an image output device 1222;

The simulator 1211 is used to obtain the control signal output by the control system to be tested, and determine the corresponding scene image according to the control signal and the virtual scene model; wherein, the virtual scene model includes a plurality of scene elements;

The image output device 1212 is configured to acquire the scene image, and output at least two scene images according to the scene image;

Wherein, the virtual scene model includes a plurality of scene elements;

In a possible implementation manner, the scene image determined by the simulator 1211 includes the at least two scene images.

In a possible implementation manner, the emulator 1211 is specifically used for:

The control signal output by the control system to be tested is acquired, and at least two scene images are determined according to the control signal, the virtual scene model and the stereo vision parameters.

In a possible implementation manner, when the simulator 1211 determines at least two scene images according to the control signal, the virtual scene model and the stereo vision parameters, it is specifically used for:

determining the relative pose of the movable platform and the scene element according to the control signal and the virtual scene model;

In a possible implementation manner, the image output device 1212 includes an image output interface, and the emulator 1211 is further used for:

Sending the determined at least two scene images to the image output interface, so that the image output interface sends the at least two scene images to the control system.

In a possible implementation manner, when the simulator 1211 sends the determined at least two scene images to the image output interface, it is specifically used for:

Sending at least two scene images in the preset format to the image output interface.

In a possible implementation manner, the image output device 1212 includes a display device for displaying the at least two scene images;

The simulator 1211 is further configured to: send the at least two scene images to a display device, so as to display the at least two scene images through the display device, so that the control system captures the scene images according to the at least two visual sensors. The image and the preset control model determine the corresponding control signal;

Correspondingly, when the emulator 1211 sends the at least two scene images to a display device to display the at least two scene images through the display device, it is specifically used for:

In a possible implementation manner, the emulator 1211 is also used for:

In a possible implementation manner, when the simulator 1211 determines the corresponding scene image according to the control signal and the virtual scene model, it is specifically used for:

At least two scene images are determined according to the control signal, the virtual scene model, the stereo vision parameters, and the calibration parameters corresponding to the vision sensor.

In a possible implementation manner, the emulator 1211 is also used for:

In a possible implementation manner, the display device is a 3D display device, and the 3D display device displays the at least two scene images in a 3D projection manner.

In a possible implementation manner, the scene image determined by the simulator 1211 is a single scene image, and the image output device 1212 is specifically configured to output the at least two scene images according to the single scene image.

In a possible implementation manner, the emulator 1211 is also used for:

In a possible implementation manner, the sensing information includes point cloud data corresponding to the scene element; when the simulator 1211 determines the corresponding sensing information according to the control signal and the virtual scene model, Specifically for:

In a possible implementation manner, the emulator 1211 is also used for:

The simulation testing system provided in this embodiment can be used to execute the simulation testing methods described in the embodiments shown in FIG. 3A to FIG. 10 , and the specific implementation principles and effects can be referred to the foregoing embodiments, which will not be repeated here.

FIG. 13 is a schematic structural diagram of an emulator provided by an embodiment of the present application. As shown in FIG. 13 , the emulator includes: a memory 1301 and at least one processor 1302;

The memory 1301 stores computer-executed instructions;

The at least one processor 1302 executes the computer-executable instructions stored in the memory 1301, so that the at least one processor 1302 executes the method described in any of the foregoing embodiments.

Optionally, the above-mentioned memory 1301 may be independent or integrated with the processor 1302 .

When the memory 1301 is set independently, the emulator may further include a bus for connecting the memory 1301 and the processor 1302.

An embodiment of the present application further provides a computer-readable storage medium, where a computer program is stored on the readable storage medium; when the computer program is executed, the method described in any of the foregoing embodiments is implemented.

An embodiment of the present application also provides a computer program product, including a computer program, which implements the method described in any of the foregoing embodiments when the computer program is executed by a processor.

Those of ordinary skill in the art can understand that all or part of the steps of implementing the above method embodiments can be completed by program instructions related to hardware, the aforementioned program can be stored in a computer-readable storage medium, and when the program is executed, execute Including the steps of the above-mentioned method embodiments; and the aforementioned storage medium includes: read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disks or optical disks and other various programs that can store program codes medium.

In the several embodiments provided in this application, it should be understood that the disclosed apparatus and method may be implemented in other manners. For example, the device embodiments described above are only illustrative. For example, the division of the modules is only a logical function division. In actual implementation, there may be other division methods. For example, multiple modules may be combined or integrated. to another system, or some features can be ignored, or not implemented.

The above-mentioned integrated modules implemented in the form of software functional modules may be stored in a computer-readable storage medium. The above-mentioned software function modules are stored in a storage medium, and include several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor to execute part of the steps of the methods described in the various embodiments of the present application.

It should be understood that the above-mentioned processor may be a central processing unit (Central Processing Unit, referred to as CPU), or other general-purpose processors, digital signal processors (Digital Signal Processor, referred to as DSP), application specific integrated circuit (Application Specific Integrated Circuit, Referred to as ASIC) and so on. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in conjunction with the invention can be directly embodied as executed by a hardware processor, or executed by a combination of hardware and software modules in the processor.

The memory may include high-speed RAM memory, and may also include non-volatile storage NVM, such as at least one magnetic disk memory, and may also be a U disk, a removable hard disk, a read-only memory, a magnetic disk or an optical disk, and the like.

An exemplary storage medium is coupled to the processor, such that the processor can read information from, and write information to, the storage medium. Of course, the storage medium can also be an integral part of the processor. The processor and the storage medium may be located in Application Specific Integrated Circuits (ASIC for short). Of course, the processor and the storage medium may also exist in the electronic device or the host device as discrete components.

It should be noted that, herein, the terms "comprising", "comprising" or any other variation thereof are intended to encompass non-exclusive inclusion, such that a process, method, article or device comprising a series of elements includes not only those elements, It also includes other elements not expressly listed or inherent to such a process, method, article or apparatus. Without further limitation, an element qualified by the phrase "comprising a..." does not preclude the presence of additional identical elements in a process, method, article or apparatus that includes the element.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, but not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: The technical solutions described in the foregoing embodiments can still be modified, or some or all of the technical features thereof can be equivalently replaced; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the technical solutions of the embodiments of the present application. scope.

Claims

A simulation test method, characterized in that the method is applied to a simulation test system; the simulation test system is used to test a control system of a movable platform based on a virtual scene model, and the scene model includes a plurality of scene elements;

The control system is configured to, based on scene images observed by at least two visual sensors, output a control signal for controlling the movable platform; the method includes:

acquiring the control signal of the movable platform output by the control system;

Based on the control signal, simulate the movement of the movable platform in the scene model to obtain the relative pose between the movable platform and the scene element;

generating a plurality of scene images according to the relative pose, the plurality of scene images including scene images observed by at least two of the vision sensors when the movable platform moves in the scene model;

The plurality of scene images are output, so that the control system generates corresponding control signals according to the plurality of scene images.
The method according to claim 1, wherein generating a plurality of scene images according to the relative pose, comprising:

According to the relative pose between the movable platform and the scene element, an imaging change of the scene element observable by the movable platform is determined, and a plurality of scene images are generated based on the imaging change.
The method according to claim 1, wherein generating a plurality of scene images according to the relative pose, comprising:

The plurality of scene images are generated according to the relative pose and stereo vision parameters between the movable platform and the scene elements.
The method according to claim 3, wherein the stereo vision parameter is determined by the installation pose of the at least two vision sensors and/or the relative pose between the at least two vision sensors.
The method according to claim 1, wherein outputting the plurality of scene images comprises:

Sending the plurality of scene images to the control system through an image output interface.
The method according to claim 5, wherein the plurality of scene images are sent to the control system through an image output interface, comprising:

converting the plurality of scene images into a plurality of scene images in a preset format;

Sending the plurality of scene images in the preset format to the control system through an image output interface.
The method according to claim 1, wherein outputting the plurality of scene images, so that the control system generates corresponding control signals according to the plurality of scene images, comprises:

Send the plurality of scene images to a display device to display the plurality of scene images through the display device, so that the control system outputs corresponding images based on the at least two visual sensors photographing the display device. control signal;

The number of the visual sensors is the same as the number of displayed scene images; the at least two visual sensors are in one-to-one correspondence with the plurality of scene images, and the visual sensors are used to photograph the corresponding scene images.
The method according to claim 7, wherein the number of the display devices is the same as the number of the visual sensors, and at least two of the display devices are in one-to-one correspondence with the at least two visual sensors, and each visual sensor is in one-to-one correspondence. The sensor is used to shoot the picture displayed by the corresponding display device;

Correspondingly, sending the plurality of scene images to a display device, so as to display the plurality of scene images through the display device, includes:

Each scene image is sent to its corresponding display device for display, so as to display the plurality of scene images through at least two of the display devices.
The method according to claim 8, wherein before sending each scene image to its corresponding display device for display, the method further comprises:

Image conversion is performed on at least part of the multiple scene images according to the calibration parameters corresponding to the visual sensor.
The method according to claim 8, wherein generating a plurality of scene images according to the relative pose, comprising:

A plurality of scene images are generated according to the relative poses between the movable platform and the scene elements, stereo vision parameters, and calibration parameters corresponding to the vision sensor.
The method according to claim 9 or 10, further comprising:

The vision sensor is calibrated by a camera calibration method to determine the calibration parameters of the vision sensor.
The method according to claim 7, wherein the display device is a 3D display device;

Sending the plurality of scene images to a display device to display the plurality of scene images through the display device, including:

Sending the plurality of scene images to the 3D display device, so that the 3D display device displays the plurality of scene images through 3D projection.
The method according to any one of claims 1-10, wherein the method further comprises:

Determine corresponding sensing information according to the relative pose between the movable platform and the scene element;

The sensing information is output, so that the control system determines a corresponding control signal according to the plurality of scene images and the sensing information.
The method according to claim 13, wherein the sensing information comprises point cloud data corresponding to the scene element.
The method according to any one of claims 1-10, wherein the method further comprises:

determining the operating state of the movable platform according to the control signal;

The control system is evaluated according to the operating state of the movable platform.
The method according to any one of claims 1-10, wherein the movable platform is a vehicle, and the control system is a driving control system applied to the vehicle.
A simulation testing method, comprising:

obtaining a control signal output by the control system to be tested, where the control signal is a control signal for controlling the movable platform output by the control system according to the historically input scene image and a preset control model;

outputting at least two scene images according to the control signal and the virtual scene model;

Wherein, the scene model includes multiple scene elements;

The at least two scene images include a first scene image and a second scene image, the first scene image includes a first pixel area, the second scene image includes a second pixel area, the first pixel area and the The second pixel area describes the same scene element; the position of the first pixel area in the first scene image is between the position of the second pixel area in the second scene image The deviation is determined according to the relative pose between the visual sensors of the movable platform;

The outputted multiple scene images are used to represent the movement of the movable platform in response to the control signal, and after the relative pose between the movable platform and the scene elements is changed, observed by the vision sensor of the movable platform to the scene image of the scene element.
The method according to claim 17, wherein, outputting at least two scene images according to the control signal and the virtual scene model, comprising:

determining at least two scene images according to the control signal, the virtual scene model and the stereo vision parameters;

The at least two scene images are output.
The method according to claim 18, wherein determining at least two scene images according to the control signal, the virtual scene model and the stereo vision parameters, comprising:

determining the relative pose of the movable platform and the scene element according to the control signal and the virtual scene model;

At least two scene images are determined according to the relative poses of the movable platform and the scene elements and stereo vision parameters.
The method according to claim 18, wherein the stereo vision parameter is determined by the installation pose of the vision sensor of the movable platform and/or the relative pose between the vision sensors of the movable platform.
The method according to claim 17, wherein outputting at least two scene images comprises:

The at least two scene images are sent to the control system through an image output interface.
The method according to claim 21, wherein sending the at least two scene images to the control system through an image output interface comprises:

converting the at least two scene images into at least two scene images in a preset format;

The at least two scene images in the preset format are sent to the control system through an image output interface.
The method according to claim 17, wherein outputting at least two scene images comprises:

Send the at least two scene images to a display device, so that the at least two scene images are displayed by the display device, so that the control system determines according to the images captured by the at least two visual sensors and a preset control model Corresponding control signal;

The number of the visual sensors is the same as the number of output scene images; the at least two visual sensors are in one-to-one correspondence with the at least two scene images, and the visual sensors are used to photograph the corresponding scene images.
The method according to claim 23, wherein the number of the display devices is the same as the number of the visual sensors, at least two of the display devices are in one-to-one correspondence with the at least two visual sensors, and each visual sensor is in one-to-one correspondence. The sensor is used to shoot the picture displayed by the corresponding display device;

Correspondingly, sending the at least two scene images to a display device, so as to display the at least two scene images through the display device, includes:

Each scene image is sent to its corresponding display device for display, so as to display the at least two scene images through at least two of the display devices.
The method according to claim 24, wherein before sending the scene image to the corresponding display device for display, it further comprises:

Image conversion is performed on at least part of the at least two scene images according to the calibration parameters corresponding to the visual sensor.
The method according to claim 24, wherein outputting at least two scene images according to the control signal and the virtual scene model, comprising:

Determine at least two scene images according to the control signal, the virtual scene model, the stereo vision parameters, and the calibration parameters corresponding to the vision sensor;

The at least two scene images are output.
The method of claim 25 or 26, further comprising:

The vision sensor is calibrated by a camera calibration method to determine the calibration parameters of the vision sensor.
The method according to claim 23, wherein the display device is a 3D display device;

Sending the at least two scene images to a display device to display the at least two scene images through the display device includes:

The at least two scene images are sent to the 3D display device, so that the 3D display device displays the at least two scene images by means of 3D projection.
The method according to any one of claims 17-26, wherein the method further comprises:

Determine corresponding sensing information according to the control signal and the virtual scene model;

The sensing information is output, so that the control system determines a corresponding control signal according to the at least two scene images and the sensing information.
The method according to claim 29, wherein the sensing information comprises point cloud data corresponding to the scene element;

According to the control signal and the virtual scene model, the corresponding sensing information is determined, including:

determining the relative pose between the movable platform and the scene element according to the control signal and the virtual scene model;

Point cloud data is determined according to the relative pose between the movable platform and the scene element.
The method according to any one of claims 17-26, wherein the method further comprises:

determining the operating state of the movable platform according to the control signal;

The control system is evaluated according to the operating state of the movable platform.
The method according to any one of claims 17-26, wherein the control system is a driving control system applied to a vehicle, and the movable platform is a vehicle.
A simulation test system, characterized in that the simulation test system is used to test a control system of a movable platform based on a virtual scene model, and the scene model includes a plurality of scene elements;

The control system is configured to output a control signal for controlling the movable platform based on scene images observed by at least two visual sensors;

The simulation test system includes an emulator and an image output device;

The simulator is used to obtain the control signal of the movable platform output by the control system, and based on the control signal, simulate the movement of the movable platform in the scene model, and obtain the relationship between the movable platform and the relative poses between scene elements, and generate an observable scene image based on the relative poses;

The image output device is configured to acquire the scene image generated by the simulator, and output a plurality of scene images according to the acquired scene image, and the plurality of scene images include at least two images when the movable platform moves in the scene model. scene images observed by the visual sensor, so that the control system generates corresponding control signals according to the plurality of scene images.
The system according to claim 33, wherein when the simulator generates an observable scene image based on the relative pose, it is specifically used for:

The plurality of scene images are generated based on relative poses between the movable platform and the scene elements.
The system according to claim 34, wherein when the simulator generates a plurality of scene images according to the relative pose, it is specifically used for:

According to the relative pose between the movable platform and the scene element, an imaging change of the scene element observable by the movable platform is determined, and a plurality of scene images are generated based on the imaging change.
The system according to claim 34, wherein when the simulator generates a plurality of scene images according to the relative pose, it is specifically used for:

The plurality of scene images are generated according to the relative pose and stereo vision parameters between the movable platform and the scene elements.
The system of claim 36, wherein the stereo vision parameter is determined by the installation pose of the at least two vision sensors and/or the relative pose between the at least two vision sensors.
The system according to claim 34, wherein the image output device comprises an image output interface, and the emulator is further configured to:

Sending the plurality of scene images to the image output interface, so that the image output interface sends the plurality of scene images to the control system.
The system according to claim 38, wherein when the simulator sends the multiple scene images to the image output interface, it is specifically used for:

converting the plurality of scene images into a plurality of scene images in a preset format;

Sending the plurality of scene images in the preset format to the image output interface.
The system of claim 34, wherein the image output device comprises a display device for displaying the plurality of scene images;

The simulator is further configured to: send the plurality of scene images to a display device, so as to display the plurality of scene images through the display device, so that the control system photographs the display device according to at least two visual sensors The obtained image outputs the corresponding control signal;

The number of the visual sensors is the same as the number of displayed scene images; the at least two visual sensors are in one-to-one correspondence with the plurality of scene images, and the visual sensors are used to photograph the corresponding scene images.
The system according to claim 40, wherein the number of the display devices is the same as the number of the visual sensors, and at least two of the display devices are in one-to-one correspondence with the at least two visual sensors, and each visual sensor is in one-to-one correspondence. The sensor is used to shoot the picture displayed by the corresponding display device;

Correspondingly, when the simulator sends the multiple scene images to the display device to display the multiple scene images through the display device, the emulator is specifically used for:

Each scene image is sent to its corresponding display device for display, so as to display the plurality of scene images through at least two of the display devices.
The system of claim 41, wherein the emulator is further configured to:

Before sending the scene image to the corresponding display device for display, image conversion is performed on at least part of the plurality of scene images according to the calibration parameter corresponding to the visual sensor.
The system according to claim 41, wherein when the simulator generates a plurality of scene images according to the relative pose, it is specifically used for:

A plurality of scene images are generated according to the relative poses between the movable platform and the scene elements, stereo vision parameters, and calibration parameters corresponding to the vision sensor.
The system according to claim 42 or 43, wherein the emulator is further used for:

The vision sensor is calibrated by a camera calibration method to determine the calibration parameters of the vision sensor.
The system according to claim 40, wherein the display device is a 3D display device, and the 3D display device displays the plurality of scene images by means of 3D projection.
The system of claim 40, further comprising: an optical system;

The optical system is arranged between the display device and the visual sensor, and the optical system is used to perform optical conversion on the scene image output by the display device, so that the converted scene image matches the visual field of the visual sensor. field angle.
The system according to claim 33, wherein the scene image generated by the simulator is a single scene image, and the image output device is specifically configured to output the plurality of scene images according to the single scene image.
The system according to any one of claims 33-43, wherein the emulator is further used for:

Determine corresponding sensing information according to the relative pose between the movable platform and the scene element;

The sensing information is output, so that the control system determines a corresponding control signal according to the plurality of scene images and the sensing information.
The system of claim 48, wherein the sensing information includes point cloud data corresponding to the scene elements.
The system according to any one of claims 33-43, wherein the emulator is further used for:

According to the control signal, determine the running state of the movable platform;

The control system is evaluated according to the operating state of the movable platform.
The system according to any one of claims 33-43, wherein the movable platform is a vehicle, and the control system is a driving control system applied to the vehicle.
A simulation test system, characterized in that it comprises: a simulator and an image output device;

The simulator is used to obtain the control signal output by the control system to be tested, and determine the corresponding scene image according to the control signal and the virtual scene model; wherein, the virtual scene model includes a plurality of scene elements;

The image output device is configured to acquire the scene image, and output at least two scene images according to the scene image;

Wherein, the virtual scene model includes a plurality of scene elements;

The at least two scene images include a first scene image and a second scene image, the first scene image includes a first pixel area, the second scene image includes a second pixel area, the first pixel area and the The second pixel area describes the same scene element; the position of the first pixel area in the first scene image is between the position of the second pixel area in the second scene image The deviation is determined according to the relative pose between the visual sensors of the movable platform;

The control signal is a control signal for controlling the movable platform output by the control system according to the historically input at least two scene images and a preset control model;

The outputted multiple scene images are used to represent the movement of the movable platform in response to the control signal, and after the relative pose between the movable platform and the scene element is changed, the visual sensor of the movable platform observes to the scene image of the scene element.
The system of claim 52, wherein the scene images determined by the simulator include the at least two scene images.
The system according to claim 53, wherein the emulator is specifically used for:

The control signal output by the control system to be tested is acquired, and at least two scene images are determined according to the control signal, the virtual scene model and the stereo vision parameters.
The system according to claim 54, wherein when the simulator determines at least two scene images according to the control signal, the virtual scene model and the stereo vision parameters, it is specifically used for:

determining the relative pose of the movable platform and the scene element according to the control signal and the virtual scene model;

At least two scene images are determined according to the relative poses of the movable platform and the scene elements and stereo vision parameters.
The system of claim 54, wherein the stereo vision parameter is determined by the installation pose of the vision sensors of the movable platform and/or the relative pose between the vision sensors of the movable platform.
The system according to claim 52, wherein the image output device comprises an image output interface, and the emulator is further configured to:

Sending the determined at least two scene images to the image output interface, so that the image output interface sends the at least two scene images to the control system.
The system according to claim 57, wherein when the simulator sends the determined at least two scene images to the image output interface, it is specifically used for:

converting the at least two scene images into at least two scene images in a preset format;

Sending at least two scene images in the preset format to the image output interface.
The system of claim 53, wherein the image output device comprises a display device for displaying the at least two scene images;

The simulator is further configured to: send the at least two scene images to a display device, so as to display the at least two scene images through the display device, so that the control system can perform the operation according to the images captured by the at least two visual sensors. The image and the preset control model determine the corresponding control signal;

The number of the visual sensors is the same as the number of output scene images; the at least two visual sensors are in one-to-one correspondence with the at least two scene images, and the visual sensors are used to photograph the corresponding scene images.
The system according to claim 59, wherein the number of the display devices is the same as the number of the visual sensors, at least two of the display devices are in one-to-one correspondence with the at least two visual sensors, and each visual sensor is in one-to-one correspondence. The sensor is used to shoot the picture displayed by the corresponding display device;

Correspondingly, when the emulator sends the at least two scene images to a display device to display the at least two scene images through the display device, the emulator is specifically used for:

Each scene image is sent to its corresponding display device for display, so as to display the at least two scene images through at least two of the display devices.
The system of claim 60, wherein the emulator is further configured to:

Before sending the scene image to the corresponding display device for display, image conversion is performed on at least part of the at least two scene images according to the calibration parameters corresponding to the visual sensor.
The system according to claim 60, wherein when the simulator determines the corresponding scene image according to the control signal and the virtual scene model, it is specifically used for:

At least two scene images are determined according to the control signal, the virtual scene model, the stereo vision parameters, and the calibration parameters corresponding to the vision sensor.
The system according to claim 61 or 62, wherein the emulator is further used for:

The vision sensor is calibrated by a camera calibration method to determine the calibration parameters of the vision sensor.
The system according to claim 59, wherein the display device is a 3D display device, and the 3D display device displays the at least two scene images by means of 3D projection.
The system of claim 59, further comprising: an optical system;

The optical system is arranged between the display device and the visual sensor, and the optical system is used to perform optical conversion on the scene image output by the display device, so that the converted scene image matches the visual field of the visual sensor. field angle.
The system according to claim 52, wherein the scene image determined by the simulator is a single scene image, and the image output device is specifically configured to output the at least two scene images according to the single scene image.
The system according to any one of claims 52-62, wherein the emulator is further used for:

Determine corresponding sensing information according to the control signal and the virtual scene model;

The sensing information is output, so that the control system determines a corresponding control signal according to the at least two scene images and the sensing information.
The system according to claim 67, wherein the sensing information includes point cloud data corresponding to the scene elements; and the simulator determines the corresponding sensing information according to the control signal and the virtual scene model. information, specifically for:

determining the relative pose between the movable platform and the scene element according to the control signal and the virtual scene model;

Point cloud data is determined according to the relative pose between the movable platform and the scene element.
The system according to any one of claims 52-62, wherein the emulator is further used for:

According to the control signal, determine the running state of the movable platform;

The control system is evaluated according to the operating state of the movable platform.
The system according to any one of claims 52-62, wherein the control system is a driving control system applied to a vehicle, and the movable platform is a vehicle.
An emulator, comprising: a memory and at least one processor;

the memory stores computer-executable instructions;

The at least one processor executes computer-implemented instructions stored in the memory, causing the at least one processor to perform the method of any of claims 1-32.
A computer-readable storage medium, characterized in that a computer program is stored on the readable storage medium; when the computer program is executed, the method according to any one of claims 1-32 is implemented.
A computer program product, comprising a computer program, characterized in that, when the computer program is executed by a processor, the method described in any one of claims 1-32 is implemented.