CN113589707A - Augmented reality automatic driving system testing method and device - Google Patents
Augmented reality automatic driving system testing method and device Download PDFInfo
- Publication number
- CN113589707A CN113589707A CN202110941188.XA CN202110941188A CN113589707A CN 113589707 A CN113589707 A CN 113589707A CN 202110941188 A CN202110941188 A CN 202110941188A CN 113589707 A CN113589707 A CN 113589707A
- Authority
- CN
- China
- Prior art keywords
- real
- automatic driving
- scene
- augmented reality
- simulation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000003190 augmentative effect Effects 0.000 title claims abstract description 147
- 238000012360 testing method Methods 0.000 title claims abstract description 76
- 238000004088 simulation Methods 0.000 claims abstract description 171
- 238000000034 method Methods 0.000 claims abstract description 56
- 238000013507 mapping Methods 0.000 claims description 39
- 230000007613 environmental effect Effects 0.000 claims description 22
- 238000004590 computer program Methods 0.000 claims description 16
- 238000003860 storage Methods 0.000 claims description 16
- 238000012544 monitoring process Methods 0.000 claims description 10
- 230000001133 acceleration Effects 0.000 claims description 4
- 238000004806 packaging method and process Methods 0.000 claims description 4
- 230000004888 barrier function Effects 0.000 claims 1
- 238000010586 diagram Methods 0.000 description 13
- 230000008569 process Effects 0.000 description 11
- 230000006870 function Effects 0.000 description 10
- 230000006872 improvement Effects 0.000 description 9
- 238000012545 processing Methods 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 5
- 230000003287 optical effect Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 230000004075 alteration Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000005055 memory storage Effects 0.000 description 1
- 229920001296 polysiloxane Polymers 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 238000010998 test method Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B17/00—Systems involving the use of models or simulators of said systems
- G05B17/02—Systems involving the use of models or simulators of said systems electric
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Automation & Control Theory (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The specification discloses a method and a device for testing an augmented reality automatic driving system, wherein an automatic driving system sends an instruction to real automatic driving equipment to enable the real automatic driving equipment to run in a real scene, the running state of the real automatic driving equipment when running in the real scene is monitored to serve as the real running state, then the real running state is adopted to control virtual automatic driving equipment to run in a simulation environment according to the corresponding relation between the real scene and the simulation scene, and the automatic driving system is tested according to the running state of the virtual automatic driving equipment. The method ensures that the reaction of the virtual automatic driving equipment when the virtual automatic driving equipment runs in the simulation scene is consistent with the reaction of the real automatic driving equipment when the real automatic driving equipment runs in the real scene, and the automatic driving equipment runs in the simulation scene, so that the aims of reducing the testing cost of the automatic driving system testing and improving the testing efficiency of the automatic driving system are fulfilled on the premise of ensuring the accuracy of the automatic driving system.
Description
Technical Field
The present disclosure relates to the field of automatic driving, and in particular, to a method and an apparatus for testing an automatic driving system for augmented reality.
Background
With the development of technology, automatic driving equipment is gradually popularized. In order to ensure that the automatic driving equipment can safely run on a traffic road, the automatic driving equipment with the automatic driving system needs to be comprehensively and strictly tested before the automatic driving equipment is formally on the road.
In the prior art, a test method for testing an automatic driving device equipped with an automatic driving system mainly includes a simulation test and an actual vehicle test, wherein the simulation test is divided into a software test and a hardware test, the software test is a virtual test for the automatic driving system, and the hardware test is a virtual test for the automatic driving system on an Electronic Control Unit (ECU). The real vehicle test is a test of the automatic driving system by driving the automatic driving equipment carrying the automatic driving system in a specified test site.
Although the test efficiency of the simulation test is high and the cost is low, the consistency of the test result of the simulation test and the result of the automatic driving equipment in actual driving cannot be ensured. Although real-vehicle testing can obtain the real reaction of the automatic driving equipment, the testing scene in the real-vehicle testing is too simple, and the testing cost is too high.
Therefore, how to reduce the testing cost of the automatic driving system testing and improve the testing efficiency of the automatic driving system on the premise of ensuring the testing accuracy is an urgent problem to be solved.
Disclosure of Invention
The present specification provides a method and an apparatus for testing an augmented reality autopilot system, which partially solve the above problems in the prior art.
The technical scheme adopted by the specification is as follows:
the present specification provides a method for testing an augmented reality autopilot system, comprising:
sending an instruction to real automatic driving equipment through an automatic driving system to enable the real automatic driving equipment to run in a real scene;
monitoring a driving state of the real automatic driving equipment when the real automatic driving equipment drives in a real scene to be used as a real driving state;
mapping the real scene to a pre-constructed simulation scene of augmented reality, and determining a mapping relation between the real scene and the simulation scene of the augmented reality;
controlling the virtual automatic driving equipment in the simulation scene of the augmented reality to run by adopting a monitored real running state according to the mapping relation between the real scene and the simulation scene of the augmented reality;
and testing the automatic driving system according to the running state of the virtual automatic driving equipment in the augmented reality simulation scene.
Optionally, sending an instruction to the real autopilot device through an autopilot system specifically includes:
sensing environmental information in a specified range of real automatic driving equipment according to sensing equipment arranged on the real automatic driving equipment;
and sending an instruction to the real automatic driving equipment through an automatic driving system according to the sensed environmental information.
Optionally, according to the mapping relationship between the real scene and the simulation scene of augmented reality, the method further includes, before controlling the virtual automatic driving device in the simulation scene of augmented reality to drive, using the monitored real driving state:
determining a relative position of a real obstacle in a real scene to the real autonomous driving device;
determining a position corresponding to the position of the real obstacle in the augmented reality simulation scene as a simulation obstacle position according to the mapping relation between the real scene and the augmented reality simulation scene and the relative position between the real obstacle and the real automatic driving equipment;
and adding a simulation obstacle in the simulation scene of the augmented reality according to the position of the simulation obstacle.
Optionally, sending an instruction to the real autopilot device through an autopilot system specifically includes:
determining a relative position of a virtual obstacle and a virtual autopilot device in the augmented reality simulation scene as a virtual relative position;
determining the relative position of the real automatic driving equipment and the virtual obstacle in the real scene as a real relative position according to the simulation relative position and the corresponding relation between the real scene and the augmented reality simulation scene;
packaging the real relative position as environmental information sensed by sensing equipment installed on the real automatic driving equipment;
and sending an instruction to the real automatic driving equipment through an automatic driving system according to the sensed environmental information.
Optionally, the driving state includes: at least one of a speed when the autonomous device is running, an acceleration when the autonomous device is running, and a posture when the autonomous device is running.
Optionally, monitoring a driving state of the real autopilot device during driving in a real scene, as a real driving state, specifically includes:
monitoring the position of the real automatic driving equipment when driving in a real scene as a real position;
and determining the driving state of the real automatic driving equipment at the real position when the real automatic driving equipment drives in the real scene as the real driving state of the real position.
Optionally, controlling, by using the monitored real driving state according to the corresponding relationship between the real scene and the augmented reality simulation scene, driving of the virtual automatic driving device in the augmented reality simulation scene, specifically including:
for each real position, determining a position corresponding to the real position in the simulation scene of the augmented reality according to the corresponding relation between the real scene and the simulation scene of the augmented reality, and taking the position as the simulation position of the real position;
determining a real driving state of a real position of the virtual automatic driving equipment corresponding to the simulation position in the augmented reality simulation scene as a virtual driving state of the simulation position for each simulation position;
and controlling the virtual automatic driving equipment in the simulation scene of the augmented reality to drive according to the virtual driving state of each simulation position.
This specification provides an augmented reality's autopilot system testing arrangement, includes:
the driving module is used for sending an instruction to real automatic driving equipment through an automatic driving system so that the real automatic driving equipment drives in a real scene; monitoring a driving state of the real automatic driving equipment when the real automatic driving equipment drives in a real scene to be used as a real driving state;
the simulation module is used for mapping the real scene to a pre-constructed simulation scene of augmented reality and determining the mapping relation between the real scene and the simulation scene of the augmented reality; controlling the virtual automatic driving equipment in the simulation scene of the augmented reality to run by adopting a monitored real running state according to the mapping relation between the real scene and the simulation scene of the augmented reality;
and the test module is used for testing the automatic driving system according to the running state of the virtual automatic driving equipment in the augmented reality simulation scene.
The present specification provides a computer-readable storage medium storing a computer program which, when executed by a processor, implements the augmented reality autopilot system testing method described above.
The specification provides an unmanned device, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor executes the program to realize the augmented reality automatic driving system testing method.
The technical scheme adopted by the specification can achieve the following beneficial effects:
in the augmented reality autopilot system testing method provided by the specification, an instruction is sent to a real autopilot device through an autopilot system, so that the real autopilot device runs in a real scene, a running state of the real autopilot device during running in the real scene is monitored and used as a real running state, then the virtual autopilot device is controlled to run in a simulation environment by adopting the real running state according to a mapping relation between the real scene and an augmented reality simulation scene, and the autopilot system is tested according to the running state of the virtual autopilot device.
According to the method, the real driving state of the real automatic driving equipment in the real scene is adopted, and the obtained driving state data is the real driving state data of the real automatic driving equipment in the actual driving process, so that the method can ensure the accuracy of the automatic driving system test, and the automatic driving equipment drives in the simulation scene, and even if the automatic driving equipment is collided and damaged in the simulation scene, the use of the real automatic driving equipment and facilities of the real scene cannot be influenced, therefore, the method can repeatedly reappear various traffic scenes in a short time, and achieves the purposes of reducing the test cost of the automatic driving system test and improving the test efficiency of the automatic driving system test.
Drawings
The accompanying drawings, which are included to provide a further understanding of the specification and are incorporated in and constitute a part of this specification, illustrate embodiments of the specification and together with the description serve to explain the specification and not to limit the specification in a non-limiting sense. In the drawings:
FIG. 1 is a schematic flow chart of a method for testing an augmented reality autopilot system according to the present disclosure;
FIG. 2 is a schematic diagram of obstacle information sent by an electronic device to an autopilot system in this specification;
FIG. 3 is a schematic diagram of an augmented reality autopilot system testing apparatus provided herein;
fig. 4 is a schematic diagram of an electronic device corresponding to fig. 1 provided in the present specification.
Detailed Description
In order to make the objects, technical solutions and advantages of the present disclosure more clear, the technical solutions of the present disclosure will be clearly and completely described below with reference to the specific embodiments of the present disclosure and the accompanying drawings. It is to be understood that the embodiments described are only a few embodiments of the present disclosure, and not all embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments in the present specification without any creative effort belong to the protection scope of the present specification.
The technical solutions provided by the embodiments of the present description are described in detail below with reference to the accompanying drawings.
Fig. 1 is a schematic flow chart of a method for testing an augmented reality autopilot system in this specification, which specifically includes the following steps:
s100: and sending an instruction to the real automatic driving equipment through an automatic driving system, so that the real automatic driving equipment runs in a real scene.
In the prior art, although the simulation test in the automatic driving system test can utilize the control instruction data of the real automatic driving equipment during actual driving, which is obtained during the historical real vehicle test, to virtually test the automatic driving equipment, the principle is that the automatic driving system sends an instruction to the virtual automatic driving equipment to obtain the data corresponding to the driving reaction of the virtual automatic driving equipment in response to the received instruction, so as to test the automatic driving system. But for the same control instruction, the reaction of the virtual autopilot device to this control instruction differs from the reaction of the real autopilot device to this control instruction.
The method combines a real scene with an augmented reality simulation scene, so that real automatic driving equipment can run in various traffic scenes in the augmented reality simulation scene, namely, the reaction of the real automatic driving equipment is synchronized to virtual automatic driving equipment in the augmented reality simulation scene, thus not only ensuring the accuracy of data, but also improving the efficiency of testing, wherein the augmented reality simulation scene refers to the fact that real environment information such as obstacles in the real scene is fused with virtual environment information such as virtual obstacles in the simulation scene, and the augmented reality simulation scene containing various environment information in the real scene is obtained.
According to the method, a real scene and an augmented reality simulation scene need to be combined through the electronic device, and the real automatic driving device and the real obstacle in the real scene are both mapped into the augmented reality simulation scene, so that the execution main body of the method can be the electronic device which can operate the augmented reality simulation scene and can control the automatic driving device.
The automatic driving device referred to in this specification may refer to a device capable of realizing automatic driving, such as an unmanned vehicle, a robot, and an automatic distribution device. Based on this, the unmanned device to which the method for testing the augmented reality automatic driving system provided by the present specification is applied can be used for executing delivery tasks in the delivery field, such as business scenes of delivery such as express delivery, logistics, takeaway and the like by using the unmanned device.
When starting to test the automatic driving system loaded on the automatic driving device, a tester can place the real automatic driving device in a real scene, and send an instruction to the automatic driving device through the automatic driving system to enable the real automatic driving device to run in a real environment, wherein the real scene is an environment where any place where the real automatic driving device can run is located, for example: the environment of the open closed test site is not limited in this specification.
The real automatic driving equipment is provided with sensing equipment such as a radar and a camera, the sensing equipment can sense all environment information within a specified range of the automatic driving equipment, and the electronic equipment sends an instruction to the real automatic driving equipment according to the sensed surrounding environment information through an automatic driving system so that the real automatic driving equipment runs in a real scene.
Specifically, the automatic driving system sends a starting instruction to sensing equipment installed on the real automatic driving equipment, the sensing equipment responds to the received starting instruction, senses environment information in a specified range of the real automatic driving equipment and sends the sensed environment information to the automatic driving system, and the automatic driving system can enable the real automatic driving equipment to run in a real scene according to the received environment information, wherein the environment information sensed by the sensing equipment is environment information in the real scene and can be sensed by the sensing equipment in a real way.
When the real automatic driving equipment senses that a real obstacle exists in the designated range of the real automatic driving equipment in the driving process, the sensing equipment can determine the position of the real obstacle, the position information of the real obstacle is used as sensed environment information and sent to the automatic driving system, and the automatic driving system sends an instruction to the real automatic driving equipment according to the received environment information so that the real automatic driving equipment can avoid the obstacle of the real obstacle in the driving process.
S102: and monitoring the running state of the real automatic driving equipment when running in a real scene as a real running state.
After the real automatic driving device starts to drive in the real scene, the electronic device can monitor the driving state of the real automatic driving device in real time when the real automatic driving device drives in the real scene, and the driving state is used as the real driving state.
The electronic device may determine a location at which the real autonomous driving device is located at each time during the driving, and a driving state of the real autonomous driving device at the location at the time, wherein the driving state includes at least one of a speed, an acceleration, and an attitude while the autonomous driving device is driving. Specifically, the electronic device may monitor, in real time, a position of the real autopilot device when driving in a real scene as a real position, and for each real position, the electronic device may determine, as a real driving state of the real autopilot device, a driving state of the real autopilot device when driving in the real position.
For example, a real autopilot device is driven to coordinates (x)1，y1) The driving posture of the real autopilot device is straight and the driving speed is 30 km/h, and the electronic device can determine that the real autopilot device is at the real position (x)1，y1) In this case, the driving state of the real position is straight and the driving speed is 30 km/h.
S104: and mapping the real scene to a pre-constructed simulation scene of augmented reality, and determining the mapping relation between the real scene and the simulation scene of the augmented reality.
As can be seen from step S100, the method combines the real scene with the augmented reality simulation scene, and therefore, a mapping relationship between the real scene and the augmented reality simulation scene needs to be determined.
In this specification, the basis for establishing a mapping relationship between a real scene and an augmented reality simulation scene is to establish a mapping relationship between the position of a real autopilot device in the real scene and the position of a virtual autopilot device in the augmented reality simulation scene, and specifically, it is assumed that the position of the autopilot device in the real scene is t1In an attitude of R1The position of the virtual autopilot device in the augmented reality simulation scenario is t2In an attitude of R2Then the mapping relationship between the real automatic driving equipment and the virtual automatic driving equipment is
Therefore, after the position of the real automatic driving equipment is determined, the position of the virtual automatic driving equipment in the augmented reality simulation scene can be obtained according to the mapping relation.
Other methods may be adopted to establish the mapping relationship between the real scene and the augmented reality simulation scene, and this specification is not limited.
S106: and controlling the virtual automatic driving equipment in the simulation scene of the augmented reality to run by adopting the monitored real running state according to the corresponding relation between the real scene and the simulation scene of the augmented reality.
The electronic device monitors the driving state of the real autopilot device in the real scene in real time in step S102, and the electronic device may synchronize the monitored driving state of the real autopilot device with the virtual autopilot device in the augmented reality simulation scene, so that the virtual autopilot device adopts the driving state of the real autopilot device to drive in the augmented reality simulation scene, which may be understood as keeping the reaction of the virtual autopilot device when driving in the augmented reality simulation scene consistent with the reaction of the real autopilot device in the real scene.
Specifically, as can be seen from step S102, in the course of the real automatic driving device driving in the real scene, the electronic device may determine a plurality of real positions as the real automatic driving device drives, and for each real position, according to the mapping relationship between the real scene and the augmented reality simulation scene, in the augmented reality simulation scene, the electronic device may determine a position corresponding to the real position as the simulation position of the real position. For each simulated location, the electronic device may determine a real driving state of the virtual autopilot device at a real location in the augmented reality simulated scene that corresponds to the simulated location as a virtual driving state of the simulated location. And controlling the virtual automatic driving equipment in the augmented reality simulation scene to drive according to the virtual driving state of each simulation position, namely controlling the driving state of the virtual automatic driving equipment in the augmented reality simulation scene to be consistent with the driving state of the real automatic driving equipment in the position of the position in the real scene when the virtual automatic driving equipment in the augmented reality simulation scene drives to the position for each position.
Along with the above example, the electronic device may determine the real location (x) according to a mapping relationship of the real scene and the augmented reality simulation scene1，y1) Corresponding position (x) in a simulated scene of augmented reality1’，y1') the simulated position of the real position is (x)1’，y1') the electronic device may control the virtual autopilot device when the virtual autopilot device is in the emulation position(x1’，y1') the running state is straight running and the running speed is 30 km/h.
S108: and testing the automatic driving system according to the running state of the virtual automatic driving equipment in the augmented reality simulation scene.
In step S106, the electronic device may control the driving state of the virtual autopilot device using the monitored driving state of the real autopilot device in the real scene, and the electronic device may test the autopilot device according to the driving state of the virtual autopilot device in the augmented reality simulation scene.
Specifically, for each traffic scene in the augmented reality simulation scene, the electronic device determines whether the automatic driving system is qualified according to the driving state of the virtual automatic driving device in the traffic scene and the expected driving state of the automatic driving device in the traffic scene. For example, whether the automatic driving system can avoid the obstacle is tested, the electronic device controls the virtual automatic driving device to avoid the obstacle in the augmented reality simulation scene according to the driving state of the real automatic driving device, when the virtual automatic driving device collides with the obstacle in the augmented reality simulation scene, the automatic driving system is unqualified, and when the virtual automatic driving device successfully avoids the obstacle in the simulation scene, the automatic driving system is qualified.
It can be seen from the above method that the method adopts the real driving state of the real autopilot device in the real scene, and the obtained driving state data is the data of the real reaction of the real autopilot device in the actual driving process, even if the reaction of the virtual autopilot device in the augmented reality simulation scene driving is consistent with the reaction of the real autopilot device in the real scene driving, therefore, the method can ensure the accuracy of the test of the autopilot system, and the autopilot device is driven in the augmented reality simulation scene, even if the autopilot device is damaged by collision or the like in the augmented reality simulation scene, the use of the real autopilot device and the facilities of the real scene will not be affected, for example, the test autopilot device can not be controlled by the autopilot device to avoid virtual obstacles in the augmented reality simulation scene, resulting in a severe collision of the virtual autopilot device with a virtual obstacle in the augmented reality simulation scenario, while the real autopilot device in the real scenario is actually not colliding with any obstacle. Therefore, the method can reproduce various traffic scenes for many times in a short time, and achieves the purposes of reducing the test cost of the test of the automatic driving system and improving the test efficiency of the test of the automatic driving system.
Furthermore, in addition to the fact that the real automatic driving system senses a real obstacle according to the sensing device and sends an instruction to the automatic driving device, the automatic driving device can avoid the obstacle of the real obstacle, a tester can add a virtual obstacle to the augmented reality simulation scene according to a test requirement, and the automatic driving device can avoid the obstacle of the virtual obstacle in the augmented reality simulation scene by a method of packaging the relative position of the real automatic driving device and the virtual obstacle into the environmental information sensed by the sensing device.
Specifically, the electronic device may determine the position of the real autopilot device first, and determine, according to the mapping relationship between the real scene and the augmented reality simulation scene in step S104, a position corresponding to the position of the real autopilot device in the augmented reality simulation scene as the position of the virtual autopilot device, so as to map the real autopilot device into the augmented reality simulation environment.
After the position of the virtual automatic driving equipment in the augmented reality simulation scene is determined, the electronic equipment can determine the relative position between the virtual obstacle added by the tester in the augmented reality simulation environment and the virtual automatic driving equipment as a virtual relative position, and according to the determined virtual relative position and the mapping relation between the real scene and the augmented reality simulation scene, the electronic equipment can determine the relative position between the real automatic driving equipment and the virtual obstacle in the real scene as a real relative position.
The electronic equipment can package the relative position of the real automatic driving equipment and the virtual obstacle as environmental information sensed by the sensing equipment and send the environmental information to the automatic driving system, and the automatic driving system sends an instruction to the real automatic driving equipment according to the received environmental information, so that the aim of avoiding the obstacle of the real automatic driving equipment on the virtual obstacle in the augmented reality simulation scene is fulfilled. The virtual obstacle is only present in the augmented reality simulation environment, and is not a real obstacle present in a real scene, and the virtual obstacle may be a static obstacle, or may also be a dynamic obstacle such as a pedestrian, a vehicle, and the like, which is not limited in this specification.
Because the virtual obstacle does not exist in the real scene, the information of the virtual obstacle is not really sensed by the sensing equipment, but the electronic equipment encapsulates the relative position of the virtual obstacle and the real automatic driving equipment into the environmental information and sends the environmental information to the automatic driving system, so that the automatic driving system mistakenly considers that the received environmental information is the environmental information sensed by the sensing equipment, and the automatic driving system can send an instruction to the real automatic driving equipment according to the received environmental information, thereby achieving the purpose that the real automatic driving equipment can avoid the obstacle of the virtual obstacle.
It is worth noting that the real automatic driving device actually runs in the real scene, in order to ensure the running safety of the real automatic driving device in the real scene, the automatic driving device still needs to avoid obstacles in the real scene, and because the virtual automatic driving device in the augmented reality simulation scene runs according to the running state of the real automatic driving device, the electronic device can map the real obstacles in the real scene into the augmented reality simulation scene, and add the simulated obstacles corresponding to the real obstacles in the augmented reality simulation scene.
Specifically, the electronic device may sense a real obstacle in a real scene through a sensing device on the real autopilot device, so as to determine a relative position between the real obstacle and the real autopilot device, and attributes such as a shape and a size of the real obstacle. The electronic device can determine a position corresponding to the position of the real obstacle in the augmented reality simulation scene as the position of the simulated obstacle according to the relative position of the real obstacle and the real automatic driving device and the mapping relation between the real scene and the augmented reality simulation scene, and the electronic device can add the simulated obstacle in the augmented reality simulation scene according to the position of the simulated obstacle. The simulated obstacle is an obstacle in a simulated scene in which the real obstacle is mapped to the augmented reality by the electronic device and exists in reality, and the virtual obstacle is an obstacle in the simulated scene in which only the augmented reality exists.
In order to avoid a conflict generated when the automatic driving system receives the environment information sent by the sensing device and the environment information sent by the electronic device at the same time (the environment information sent by the sensing device is the real obstacle information really perceived by the sensing device, and the environment information sent by the electronic device is the information of the virtual obstacle in the augmented reality simulation scene), therefore, as shown in fig. 2, the electronic device can merge the virtual obstacle in the augmented reality simulation scene and the simulated obstacle corresponding to the real obstacle in the augmented reality simulation scene, update the augmented reality simulation scene, so as to obtain the augmented reality simulation scene containing both the virtual obstacle and the simulated obstacle, and enable the automatic driving system to receive only the environment information sent by the electronic device, wherein the environment information is formed by encapsulating the obstacle (including the simulated obstacle and the virtual obstacle) in the augmented reality simulation scene and the relative position of the real automatic driving device And (4) information. Because the information of the simulated obstacle contained in the environmental information sent by the electronic device is consistent with the information of the real obstacle sensed by the sensing device, the automatic driving device with the automatic driving system can avoid the real obstacle in the real scene and can avoid the virtual obstacle in the augmented reality simulated scene.
Based on the same idea, the augmented reality automatic driving system testing method provided in one or more embodiments of the present specification further provides a corresponding augmented reality automatic driving system testing apparatus, as shown in fig. 3.
Fig. 3 is a schematic diagram of an augmented reality autopilot system testing apparatus provided in this specification, which specifically includes:
a driving module 301, a simulation module 302, and a test module 303, wherein:
the driving module 301 is configured to send an instruction to a real autopilot device through an autopilot system, so that the real autopilot device drives in a real scene; monitoring a driving state of the real automatic driving equipment when the real automatic driving equipment drives in a real scene to be used as a real driving state;
a simulation module 302, configured to map the real scene into a pre-constructed augmented reality simulation scene, and determine a mapping relationship between the real scene and the augmented reality simulation scene; controlling the virtual automatic driving equipment in the simulation scene of the augmented reality to run by adopting a monitored real running state according to the mapping relation between the real scene and the simulation scene of the augmented reality and the mapping relation between the real scene and the simulation scene of the augmented reality;
a testing module 303, configured to test the autopilot system according to a driving state of the virtual autopilot device in the augmented reality simulation scene.
Optionally, the driving module 301 is specifically configured to sense environmental information within a specified range of a real automatic driving device according to a sensing device installed on the real automatic driving device; and sending an instruction to the real automatic driving equipment through an automatic driving system according to the sensed environmental information.
Optionally, the simulation module 302 is further configured to determine a relative position of a real obstacle in a real scene to the real autonomous driving device; determining a position corresponding to the position of the real obstacle in the augmented reality simulation scene as a simulation obstacle position according to the mapping relation between the real scene and the augmented reality simulation scene and the relative position between the real obstacle and the real automatic driving equipment; and adding a simulation obstacle in the simulation scene of the augmented reality according to the position of the simulation obstacle.
Optionally, the driving module 301 is specifically configured to determine a relative position between a virtual obstacle and a virtual automatic driving device in the augmented reality simulation scene as a virtual relative position; determining the relative position of the real automatic driving equipment and the virtual obstacle in the real scene as a real relative position according to the simulation relative position and the mapping relation between the real scene and the augmented reality simulation scene; packaging the real relative position as environmental information sensed by sensing equipment installed on the real automatic driving equipment; and sending an instruction to the real automatic driving equipment through an automatic driving system according to the sensed environmental information.
Optionally, the driving state includes: at least one of a speed when the autonomous device is running, an acceleration when the autonomous device is running, and a posture when the autonomous device is running.
Optionally, the driving module 301 is specifically configured to monitor a position of the real automatic driving device during driving in a real scene as a real position; and determining the driving state of the real automatic driving equipment at the real position when the real automatic driving equipment drives in the real scene as the real driving state of the real position.
Optionally, the simulation module 302 is specifically configured to, for each real position, determine, according to a mapping relationship between the real scene and the augmented reality simulation scene, a position in the augmented reality simulation scene corresponding to the real position as a simulation position of the real position; determining a real driving state of a real position of the virtual automatic driving equipment corresponding to the simulation position in the augmented reality simulation scene as a virtual driving state of the simulation position for each simulation position; and controlling the virtual automatic driving equipment in the simulation scene of the augmented reality to drive according to the virtual driving state of each simulation position.
The present specification also provides a computer-readable storage medium storing a computer program, which can be used to execute the augmented reality autopilot system testing method provided in fig. 1 above.
This specification also provides a schematic block diagram of the electronic device shown in fig. 4. As shown in fig. 4, the drone includes, at the hardware level, a processor, an internal bus, a network interface, a memory, and a non-volatile memory, although it may also include hardware required for other services. The processor reads a corresponding computer program from the nonvolatile memory into the memory and then runs the computer program to implement the augmented reality autopilot system testing method described in fig. 1. Of course, besides the software implementation, the present specification does not exclude other implementations, such as logic devices or a combination of software and hardware, and the like, that is, the execution subject of the following processing flow is not limited to each logic unit, and may be hardware or logic devices.
In the 90 s of the 20 th century, improvements in a technology could clearly distinguish between improvements in hardware (e.g., improvements in circuit structures such as diodes, transistors, switches, etc.) and improvements in software (improvements in process flow). However, as technology advances, many of today's process flow improvements have been seen as direct improvements in hardware circuit architecture. Designers almost always obtain the corresponding hardware circuit structure by programming an improved method flow into the hardware circuit. Thus, it cannot be said that an improvement in the process flow cannot be realized by hardware physical modules. For example, a Programmable Logic Device (PLD), such as a Field Programmable Gate Array (FPGA), is an integrated circuit whose Logic functions are determined by programming the Device by a user. A digital system is "integrated" on a PLD by the designer's own programming without requiring the chip manufacturer to design and fabricate application-specific integrated circuit chips. Furthermore, nowadays, instead of manually making an Integrated Circuit chip, such Programming is often implemented by "logic compiler" software, which is similar to a software compiler used in program development and writing, but the original code before compiling is also written by a specific Programming Language, which is called Hardware Description Language (HDL), and HDL is not only one but many, such as abel (advanced Boolean Expression Language), ahdl (alternate Hardware Description Language), traffic, pl (core universal Programming Language), HDCal (jhdware Description Language), lang, Lola, HDL, laspam, hardward Description Language (vhr Description Language), vhal (Hardware Description Language), and vhigh-Language, which are currently used in most common. It will also be apparent to those skilled in the art that hardware circuitry that implements the logical method flows can be readily obtained by merely slightly programming the method flows into an integrated circuit using the hardware description languages described above.
The controller may be implemented in any suitable manner, for example, the controller may take the form of, for example, a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro) processor, logic gates, switches, an Application Specific Integrated Circuit (ASIC), a programmable logic controller, and an embedded microcontroller, examples of which include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320, the memory controller may also be implemented as part of the control logic for the memory. Those skilled in the art will also appreciate that, in addition to implementing the controller as pure computer readable program code, the same functionality can be implemented by logically programming method steps such that the controller is in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers, embedded microcontrollers and the like. Such a controller may thus be considered a hardware component, and the means included therein for performing the various functions may also be considered as a structure within the hardware component. Or even means for performing the functions may be regarded as being both a software module for performing the method and a structure within a hardware component.
The systems, devices, modules or units illustrated in the above embodiments may be implemented by a computer chip or an entity, or by a product with certain functions. One typical implementation device is a computer. In particular, the computer may be, for example, a personal computer, a laptop computer, a cellular telephone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.
For convenience of description, the above devices are described as being divided into various units by function, and are described separately. Of course, the functions of the various elements may be implemented in the same one or more software and/or hardware implementations of the present description.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.
Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
As will be appreciated by one skilled in the art, embodiments of the present description may be provided as a method, system, or computer program product. Accordingly, the description may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the description may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
This description may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.
The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the system embodiment, since it is substantially similar to the method embodiment, the description is simple, and for the relevant points, reference may be made to the partial description of the method embodiment.
The above description is only an example of the present specification, and is not intended to limit the present specification. Various modifications and alterations to this description will become apparent to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present specification should be included in the scope of the claims of the present specification.
Claims (10)
1. An augmented reality autopilot system testing method, comprising:
sending an instruction to real automatic driving equipment through an automatic driving system to enable the real automatic driving equipment to run in a real scene;
monitoring a driving state of the real automatic driving equipment when the real automatic driving equipment drives in a real scene to be used as a real driving state;
mapping the real scene to a pre-constructed simulation scene of augmented reality, and determining a mapping relation between the real scene and the simulation scene of the augmented reality;
controlling the virtual automatic driving equipment in the simulation scene of the augmented reality to run by adopting a monitored real running state according to the mapping relation between the real scene and the simulation scene of the augmented reality;
and testing the automatic driving system according to the running state of the virtual automatic driving equipment in the augmented reality simulation scene.
2. The method according to claim 1, wherein sending instructions to the real autopilot device via an autopilot system includes:
sensing environmental information in a specified range of real automatic driving equipment according to sensing equipment arranged on the real automatic driving equipment;
and sending an instruction to the real automatic driving equipment through an automatic driving system according to the sensed environmental information.
3. The method of claim 1, wherein the method further comprises controlling a virtual autopilot device in the simulation scenario to travel before using the monitored real travel state according to the mapping relationship between the real scenario and the simulation scenario of the augmented reality, and further comprising:
determining a relative position of a real obstacle in a real scene to the real autonomous driving device;
determining a position corresponding to the position of the real obstacle in the augmented reality simulation scene as a simulation obstacle position according to the mapping relation between the real scene and the augmented reality simulation scene and the relative position between the real obstacle and the real automatic driving equipment;
and adding a simulation obstacle in the simulation scene of the augmented reality according to the position of the simulation obstacle.
4. The method according to claim 1, wherein sending instructions to the real autopilot device via an autopilot system includes:
determining a relative position of a virtual obstacle and a virtual autopilot device in the augmented reality simulation scene as a virtual relative position;
determining the relative position of the real automatic driving equipment and the virtual barrier in the real scene as a real relative position according to the simulation relative position of the augmented reality and the mapping relation between the real scene and the simulation scene of the augmented reality;
packaging the real relative position as environmental information sensed by sensing equipment installed on the real automatic driving equipment;
and sending an instruction to the real automatic driving equipment through an automatic driving system according to the sensed environmental information.
5. The method of claim 1, wherein the driving condition comprises: at least one of a speed when the autonomous device is running, an acceleration when the autonomous device is running, and a posture when the autonomous device is running.
6. The method according to claim 1, wherein monitoring the driving state of the real autopilot device while driving in the real scene as the real driving state specifically comprises:
monitoring the position of the real automatic driving equipment when driving in a real scene as a real position;
and determining the driving state of the real automatic driving equipment at the real position when the real automatic driving equipment drives in the real scene as the real driving state of the real position.
7. The method according to claim 6, wherein controlling the virtual autopilot device in the simulation scene to travel by using the monitored real travel state according to the mapping relationship between the real scene and the simulation scene of the augmented reality comprises:
for each real position, determining a position corresponding to the real position in the simulation scene of the augmented reality according to the mapping relation between the real scene and the simulation scene of the augmented reality, and taking the position as the simulation position of the real position;
determining a real driving state of a real position of the virtual automatic driving equipment corresponding to the simulation position in the augmented reality simulation scene as a virtual driving state of the simulation position for each simulation position;
and controlling the virtual automatic driving equipment in the simulation scene of the augmented reality to drive according to the virtual driving state of each simulation position.
8. An augmented reality autopilot system testing apparatus, comprising:
the driving module is used for sending an instruction to real automatic driving equipment through an automatic driving system so that the real automatic driving equipment drives in a real scene; monitoring a driving state of the real automatic driving equipment when the real automatic driving equipment drives in a real scene to be used as a real driving state;
the simulation module is used for mapping the real scene to a pre-constructed simulation scene of augmented reality and determining the mapping relation between the real scene and the simulation scene of the augmented reality; controlling the virtual automatic driving equipment in the simulation scene of the augmented reality to run by adopting a monitored real running state according to the mapping relation between the real scene and the simulation scene of the augmented reality;
and the test module is used for testing the automatic driving system according to the running state of the virtual automatic driving equipment in the augmented reality simulation scene.
9. A computer-readable storage medium, characterized in that the storage medium stores a computer program which, when executed by a processor, implements the method of any of the preceding claims 1 to 7.
10. An unmanned aerial vehicle comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor when executing the program implements the method of any of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110941188.XA CN113589707A (en) | 2021-08-17 | 2021-08-17 | Augmented reality automatic driving system testing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110941188.XA CN113589707A (en) | 2021-08-17 | 2021-08-17 | Augmented reality automatic driving system testing method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113589707A true CN113589707A (en) | 2021-11-02 |
Family
ID=78258287
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110941188.XA Pending CN113589707A (en) | 2021-08-17 | 2021-08-17 | Augmented reality automatic driving system testing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113589707A (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111505965A (en) * | 2020-06-17 | 2020-08-07 | 深圳裹动智驾科技有限公司 | Method and device for simulation test of automatic driving vehicle, computer equipment and storage medium |
| CN111781855A (en) * | 2020-07-15 | 2020-10-16 | 北京领骏科技有限公司 | Traffic on-loop automatic driving simulation system |
| CN112198859A (en) * | 2020-09-07 | 2021-01-08 | 西安交通大学 | Method, system and device for testing automatic driving vehicle in vehicle ring under mixed scene |
| CN112965466A (en) * | 2021-02-18 | 2021-06-15 | 北京百度网讯科技有限公司 | Reduction test method, device, equipment and program product of automatic driving system |
-
2021
- 2021-08-17 CN CN202110941188.XA patent/CN113589707A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111505965A (en) * | 2020-06-17 | 2020-08-07 | 深圳裹动智驾科技有限公司 | Method and device for simulation test of automatic driving vehicle, computer equipment and storage medium |
| CN111781855A (en) * | 2020-07-15 | 2020-10-16 | 北京领骏科技有限公司 | Traffic on-loop automatic driving simulation system |
| CN112198859A (en) * | 2020-09-07 | 2021-01-08 | 西安交通大学 | Method, system and device for testing automatic driving vehicle in vehicle ring under mixed scene |
| CN112965466A (en) * | 2021-02-18 | 2021-06-15 | 北京百度网讯科技有限公司 | Reduction test method, device, equipment and program product of automatic driving system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110414098B (en) | Generation method and device of simulation test environment | |
| CN112987760B (en) | Trajectory planning method and device, storage medium and electronic equipment | |
| CN110162089B (en) | Unmanned driving simulation method and device | |
| CN111126362B (en) | Method and device for predicting obstacle track | |
| CN113296541B (en) | Future collision risk based unmanned equipment control method and device | |
| CN112327864A (en) | Control method and control device of unmanned equipment | |
| CN113419547B (en) | Multi-vehicle cooperative control method and device | |
| CN113032261B (en) | Simulation test method and device | |
| CN112799411A (en) | Control method and device of unmanned equipment | |
| CN111532285B (en) | Vehicle control method and device | |
| CN111090286A (en) | Unmanned vehicle motion state planning method and device | |
| CN112731957B (en) | Unmanned aerial vehicle control method and device, computer readable storage medium and unmanned aerial vehicle | |
| CN113968243A (en) | Obstacle trajectory prediction method, device, equipment and storage medium | |
| CN112649012A (en) | Trajectory planning method, equipment, medium and unmanned equipment | |
| CN110895406B (en) | Method and device for testing unmanned equipment based on interferent track planning | |
| CN113589707A (en) | Augmented reality automatic driving system testing method and device | |
| CN114721290A (en) | Simulation test scene generation method and device, storage medium and electronic equipment | |
| CN112462403A (en) | Positioning method, positioning device, storage medium and electronic equipment | |
| CN113340311B (en) | Path planning method and device for unmanned equipment | |
| CN114280960A (en) | Automatic driving simulation method and device, storage medium and electronic equipment | |
| CN112987762B (en) | Trajectory planning method and device, storage medium and electronic equipment | |
| CN112859883B (en) | Control method and control device of unmanned equipment | |
| CN113703352A (en) | Safety early warning method and device based on remote driving | |
| CN112684720A (en) | Simulation test method and device | |
| CN113859554A (en) | Unmanned equipment control method and device, storage medium and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20211102 |