CN116147931A

CN116147931A - Test method and device for automatic driving vehicle

Info

Publication number: CN116147931A
Application number: CN202310097314.7A
Authority: CN
Inventors: 杨开睿; 张平
Original assignee: Alibaba Damo Institute Hangzhou Technology Co Ltd
Current assignee: Alibaba Damo Institute Hangzhou Technology Co Ltd
Priority date: 2023-01-19
Filing date: 2023-01-19
Publication date: 2023-05-23

Abstract

The embodiment of the application discloses a test method and a test device for an automatic driving vehicle, and relates to the technical field of automatic driving. The method comprises the following steps: obtaining bad case data of the tested automatic driving vehicle; performing scene enumeration according to the bad case data, wherein the enumerated scenes comprise the tested automatic driving vehicle or comprise the combination of the tested automatic driving vehicle and at least one obstacle contained in the bad case data; respectively carrying out simulation test on the enumerated scenes to determine abnormal scenes with the same abnormal type as the bad case data; and determining a responsible entity of the bad case data from the automatic driving vehicle and the obstacle included in the bad case data by using the abnormal scene. The method and the device can improve accuracy and efficiency of bad case analysis and reduce labor cost.

Description

Test method and device for automatic driving vehicle

Technical Field

The present disclosure relates to the field of automatic driving technologies, and in particular, to a method and an apparatus for testing an automatic driving vehicle.

Background

The automatic driving vehicle is cooperated with a sensing sensor, an artificial intelligence, a global positioning system and the like, so that the vehicle can safely and automatically run. The automatic driving vehicle is landed and needs a large number of tests, and the algorithm iteration also needs a large number of tests, so that the simulation test can greatly accelerate the algorithm iteration efficiency and the product optimization speed. An autonomous vehicle may generate a significant amount of badcase (bad examples) during drive tests, such as sudden braking during cornering. Then, manual analysis is needed for badcase, and responsible entities are marked. However, this approach relies on human experience, is low in accuracy, high in labor cost and low in efficiency, and is limited by the problem of human efficiency, resulting in limited test coverage.

Disclosure of Invention

In view of this, the application provides a test method and device for an automatic driving vehicle, so as to improve accuracy and efficiency of badcase analysis and reduce labor cost.

The application provides the following scheme:

in a first aspect, there is provided a test method for an autonomous vehicle, the method comprising:

obtaining bad case data of the tested automatic driving vehicle;

performing scene enumeration according to the bad case data, wherein the enumerated scenes comprise the tested automatic driving vehicle or comprise the combination of the tested automatic driving vehicle and at least one obstacle contained in the bad case data;

respectively carrying out simulation test on the enumerated scenes to determine abnormal scenes with the same abnormal type as the bad case data;

and determining a responsible entity of the bad case data from the tested automatic driving vehicle and the obstacle included in the bad case data by using the abnormal scene.

According to an implementation manner in the embodiments of the present application, performing scene enumeration according to the bad case data includes:

determining an entity in a first area and/or a second area according to the bad case data, wherein the entity comprises the tested automatic driving vehicle and an obstacle, the first area is an area within a preset first distance range from the position of the tested automatic driving vehicle when an abnormality corresponding to the bad case data occurs, and the second area is a driving area determined according to the navigation path of the tested automatic driving vehicle;

and combining the entities to enumerate all scenes, wherein each combination at least comprises the tested automatic driving vehicle.

According to an implementation manner in the embodiments of the present application, the performing the simulation test on the enumerated scenes includes:

sequentially performing simulation tests on the enumerated scenes according to the sequence of the number of entities contained in the scenes from small to large, wherein the entities comprise the tested automatic driving vehicles or obstacles;

if the scene of the current simulation test has the same abnormal type as the bad case data, determining the scene of the current simulation test as an abnormal scene, and ending the simulation test on the enumerated scenes respectively; otherwise, continuing to perform simulation test on the next scene.

According to an implementation manner of the embodiment of the present application, using the abnormal scenario, determining, from the autonomous vehicle and the obstacle included in the bad case data, a responsible entity of the bad case data includes:

if the abnormal scene only comprises the tested automatic driving vehicle, determining the tested automatic driving vehicle as a responsible entity of the bad case data;

and if the abnormal scene comprises a combination of the tested automatic driving vehicle and at least one obstacle, taking the obstacle contained in the abnormal scene as a responsible entity.

According to an implementation manner in an embodiment of the present application, the method further includes:

performing generalization processing on the motion information of the responsible entity in the abnormal scene to obtain a generalization scene;

and outputting the information of the generalization scene.

According to an implementation manner of the embodiment of the present application, before outputting the information of the generalized scene, the method further includes:

performing simulation test on the generalization scene, and screening to obtain a generalization scene with the same abnormal type as the bad case data;

outputting the information of the generalization scene comprises: and outputting the information of the generalization scene obtained by screening.

According to an implementation manner in the embodiment of the present application, the exception types corresponding to the bad case data include: the risk of sudden braking, collision or running exceeds a preset risk level.

In a second aspect, there is provided a test device for an autonomous vehicle, the device comprising:

a bad case acquisition unit configured to acquire bad case data of the tested automatic driving vehicle;

the scene enumeration unit is configured to enumerate scenes according to the bad case data, wherein the enumerated scenes comprise the tested automatic driving vehicle or a combination of the tested automatic driving vehicle and at least one obstacle contained in the bad case data;

the simulation test unit is configured to perform simulation test on the enumerated scenes respectively and determine abnormal scenes with the same abnormal type as the bad case data;

and a responsibility estimating unit configured to determine a responsibility entity of the bad case data from the tested automatic driving vehicle and an obstacle included in the bad case data, using the abnormal scene.

According to a third aspect, there is provided a computer readable storage medium having stored thereon a computer program which when executed by a processor performs the steps of the method of any of the first aspects described above.

According to a fourth aspect, there is provided an electronic device comprising:

one or more processors; and

a memory associated with the one or more processors, the memory for storing program instructions that, when read for execution by the one or more processors, perform the steps of the method of any of the first aspects above.

According to a specific embodiment provided by the application, the application discloses the following technical effects:

1) According to the method and the device, scene enumeration is carried out according to the bad case data, and abnormal scenes with the same abnormal type as the bad case data are determined through simulation tests on all scenes obtained through enumeration, so that responsibility entities of the bad case data are determined through the abnormal scenes. The simulation test mode based on scene enumeration can realize automatic analysis of bad examples, does not need to rely on manual experience for analysis and labeling, reduces labor cost, improves efficiency, is easy to scale and has higher accuracy.

2) According to the method, the combination of the obstacle and the tested automatic driving vehicle is carried out in the area with certain influence on the automatic driving safety, so that scene enumeration is more efficiently carried out, and the testing efficiency is further improved.

3) According to the method and the device, the motion information of the responsible entity in the abnormal scene is subjected to generalization processing to obtain the generalization scene, the simulation test can be further carried out on the generalization scene, the generalization scene with the same abnormal type as the bad case data is obtained through screening, the information of the generalization scene is output, and the coverage of the abnormality is enlarged.

Of course, not all of the above-described advantages need be achieved at the same time in practicing any one of the products of the present application.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings that are needed in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings can be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 is a diagram of a system architecture to which embodiments of the present application are applicable;

fig. 2 is a test method for an autonomous vehicle according to an embodiment of the present application;

FIG. 3 is a schematic diagram of region division provided in an embodiment of the present application;

fig. 4a to fig. 4c are schematic diagrams of three scenario simulation tests provided in the embodiments of the present application, respectively;

FIG. 5 is a schematic block diagram of a testing device provided by an embodiment of the present application;

fig. 6 is a schematic block diagram of an electronic device provided in an embodiment of the present application.

Detailed Description

The following description of the embodiments of the present application will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are only some, but not all, of the embodiments of the present application. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments in the present application are within the scope of the protection of the present application.

The terminology used in the embodiments of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used in this application and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.

It should be understood that the term "and/or" as used herein is merely one relationship describing the association of the associated objects, meaning that there may be three relationships, e.g., a and/or B, may represent: a exists alone, A and B exist together, and B exists alone. In addition, the character "/" herein generally indicates that the front and rear associated objects are an "or" relationship.

Depending on the context, the word "if" as used herein may be interpreted as "at … …" or "at … …" or "in response to a determination" or "in response to detection". Similarly, the phrase "if determined" or "if detected (stated condition or event)" may be interpreted as "when determined" or "in response to determination" or "when detected (stated condition or event)" or "in response to detection (stated condition or event), depending on the context.

The automatic driving vehicle can generate a plurality of high-value bad examples in the real road test (road test for short), if the traditional mode is adopted for manual analysis, marking and the like, the problems of high labor cost, uncontrollable quality (depending on manual experience, which is difficult to control), low efficiency and incapability of large scale are caused. Therefore, how to efficiently and massively process the automatic driving vehicle drive test bad examples is a problem to be solved. The application provides a brand new idea.

To facilitate understanding of embodiments of the present application, a brief description of a system architecture on which embodiments of the present application are based is first provided. FIG. 1 illustrates an exemplary system architecture to which embodiments of the present application may be applied, as shown in FIG. 1, the system generally comprising a data storage device and a testing device.

The data storage device is used for storing bad case data of the tested automatic driving vehicle. The bad case data are obtained in the drive test process of the tested automatic driving vehicle, and refer to scene data of the automatic driving vehicle abnormal. The bad case data may be stored in a database or may be stored in a data storage device in the form of a data file.

The data storage means may be any device having data storage functionality. For example, the device may collect and store receipts during road testing of an automatically driven vehicle, and the device may further store data obtained by road testing of an automatically driven vehicle in a centralized manner.

The testing device in the system shown in fig. 1 is mainly used for testing the automatic driving vehicle by adopting the testing method provided by the embodiment of the application, so as to obtain the responsible entity of the bad case. The responsible entity is the entity that causes the occurrence of the bad case. In the test process, the bad examples are utilized to enumerate scenes, and the enumerated scenes are constructed and tested in a simulation environment.

The autonomous vehicle referred to in the present application is a broad expression, and may be an unmanned vehicle or an assisted driving vehicle.

The automatic driving vehicle mainly comprises a perception module, a decision module and an execution module. The sensing module senses road information, real-time traffic information, obstacle information, state information of the vehicle itself, and the like by using data acquired by a sensing sensor in the automatically driven vehicle. The decision module determines control information for the autonomous vehicle by using the information perceived by the perception module. The execution module is used for executing the control information determined by the decision module so as to realize driving control of the automatic driving vehicle. Among other things, the perception sensors in an autonomous vehicle may include image sensors, radar, infrared sensors, ultrasonic sensors, and the like. Wherein the image sensor may comprise a camera, a video camera, etc. The radar may include a lidar, a millimeter wave radar, or the like.

The test device may be any device having computing power, such as a notebook computer, a PC (Personal Computer ), or the like. Even the test equipment can be arranged at the server end to complete the test of the perception fusion system of the automatic driving vehicle.

It should be understood that the number of data storage devices and test devices in fig. 1 is merely illustrative. There may be any number of data storage devices and testing devices, as desired for implementation.

Fig. 2 is a schematic diagram of a test method for an autopilot vehicle according to an embodiment of the present application, which may be executed by a test apparatus in the system architecture shown in fig. 1. As shown in fig. 2, the method may include the steps of:

step 202: bad case data of the tested automatic driving vehicle is obtained.

Step 204: and performing scene enumeration according to the bad case data, wherein the enumerated scenes comprise the tested automatic driving vehicle or the combination of the tested automatic driving vehicle and at least one obstacle contained in the bad case data.

Step 206: and respectively carrying out simulation test on the enumerated scenes to determine the abnormal scenes with the same abnormal type as the bad case data.

Step 208: using the anomaly scenario, a responsible entity for the bad case data is determined from the tested autonomous vehicle and the obstacle contained in the bad case data.

According to the process, scene enumeration is performed according to the bad case data, and simulation tests are performed on all the enumerated scenes to determine abnormal scenes with the same abnormal type as the bad case data, so that responsibility entities of the bad case data are determined by using the abnormal scenes. The simulation test mode based on scene enumeration can realize automatic analysis of bad case data, does not need to rely on manual experience for analysis and labeling, reduces labor cost, improves efficiency, is easy to scale and has higher accuracy.

The above method provided in the embodiments of the present application is described in detail below. The above step 202, i.e. "obtaining bad case data of a tested autonomous vehicle" is described in detail below with reference to the embodiments.

The bad case data in this step is generated during the road test of the tested autonomous vehicle. And the tested automatic driving vehicle is driven on the appointed road to test whether the automatic driving vehicle can normally drive in the actual road environment. If abnormality occurs in the driving process, the abnormal data is the bad case data. The bad case data includes related data and environmental data of the tested automatic driving vehicle when the abnormality occurs. The bad case data may be data of the automatic driving vehicle itself, such as data of the automatic driving vehicle itself, road data, and data of an obstacle, etc. acquired when an abnormality occurs. The movement data of each vehicle, pedestrian, etc. may be acquired by a sensor mounted on a road or on a traffic facility, for example, a camera mounted on a traffic light bracket, a speed sensor mounted on a road side, etc.

Wherein the anomalies in the drive-test process that occur in the autonomous vehicle, i.e., the anomalies corresponding to badcase, may include, but are not limited to: sudden braking, collision, running risk exceeding a preset risk level, etc.

For example, when sudden braking occurs in an automatic driving vehicle in the drive test process, driving safety and user body feeling are generally considered to avoid the occurrence of sudden braking as much as possible, so that if sudden braking occurs, the automatic driving vehicle can be considered to be abnormal, and at the moment, self data and environment data are acquired to form bad case data of the time. Wherein sudden braking may be considered to occur when the deceleration of the autonomous vehicle is greater than or equal to a preset threshold, wherein the threshold may be set based on empirical values, trial values, etc.

For another example, the collision of the automatic driving vehicle occurs in the drive test process, and the collision is usually forbidden in the actual driving process, so that if the collision occurs, the automatic driving vehicle can be considered to be abnormal, and the self data and the environmental data are acquired to form the bad case data of this time.

As another example, automated driving vehicles are often equipped with safety personnel during drive tests, which sit in the automated driving vehicle during drive tests to address the occurrence of special situations. The running risk can be evaluated by a safety officer according to actual conditions, and if the running risk exceeds a preset risk level, the acquisition of bad case data can be triggered. Or after the drive test is finished, risk evaluation is performed on the running data of the automatic driving vehicle by the risk evaluation model, and a bad case is considered to appear under the condition that the risk evaluation exceeds a preset risk level.

The bad case data can be stored in an automatic driving vehicle, can be uploaded to a server side by the automatic driving vehicle, and can be uniformly stored in other devices with a data storage function. Accordingly, the testing device in the embodiment of the application can obtain the bad case data of the tested automatic driving vehicle from the tested automatic driving vehicle, the server side or other devices with the data storage function.

The above step 204, i.e. "scene enumeration according to bad case data", is described in detail below with reference to the embodiments.

When the scene enumeration is performed, firstly, an obstacle list is acquired according to bad case data. Since the bad case data is data and environmental data of the automatically driven vehicle acquired when the abnormality occurs, the obstacle list is actually a list of obstacles of the automatically driven vehicle when the abnormality corresponding to the bad case data occurs, that is, it is determined which obstacles are present around the automatically driven vehicle when the abnormality occurs. These obstacles may cause an abnormality to occur, but in particular, which one or more cause an abnormality to occur, i.e., are responsible entities, need to be examined. In the embodiment of the application, the investigation mode is not manual analysis, but the combination mode of the automatic driving vehicle and the obstacle enumerates different scenes, and then simulation tests are respectively carried out on the scenes to analyze.

The obstacle involved in the embodiments of the present application may include, for example, a vehicle, a pedestrian, a traffic facility, and the like, and may be any other object that has a safety hazard for the vehicle to travel, such as a tree, an animal, and the like.

Since the ability of an autonomous vehicle to collect data is relatively strong, the range of data typically collected is large. If the combination enumeration and simulation test is performed in full quantity, the calculation amount is large, and the combination enumeration and simulation test is not needed for the obstacle with lower safety influence on the automatic driving vehicle. Thus, as one of the possible ways, an entity in the first area and/or in the second area may first be determined from the bad case data, e.g. it may be determined that an obstacle in the first area and/or in the second area constitutes an obstacle list, the entity comprising the tested autonomous vehicle and the obstacle. The entities are then combined to enumerate the scenarios, where each combination includes at least the autonomous vehicle under test. For example, the autonomous vehicle under test is taken as one of the scenes, and the autonomous vehicle under test is combined with at least one obstacle in the list of obstacles, respectively.

When abnormality corresponding to the bad case data occurs, the first area is an area within a preset first distance range from the position of the tested automatic driving vehicle. That is, an area adjacent to the autonomous vehicle may be regarded as the first area. For example, a predetermined radius value may be employed, or the radius value may be determined based on speed information of the autonomous vehicle; then, a region within the above-described radius value range centered on the position of the autonomous vehicle is determined as a first region in the sensor data.

Taking fig. 3 as an example, when an abnormality corresponding to bad case data occurs, taking the position of the tested automatic driving vehicle a as the center of a circle, and taking an area within a radius range as a first area. The radius can be a preset empirical value or an experimental value, and can also be determined according to the current speed of the automatic driving vehicle. Then the obstacle in this first area is of greater impact on the safety risk of the autonomous vehicle a.

The second area is a travel area determined in accordance with a navigation path of the tested autonomous vehicle. Since the autonomous vehicle has a clear travel path, usually following a navigation route, the second region can be understood as a region which can be affected during the driving of the autonomous vehicle.

For example, a navigation path of the autonomous vehicle may be acquired, and a lane in which the autonomous vehicle is located and a region corresponding to the latest N lanes on the navigation path are determined as a driving region, where N is a positive integer.

Where the most recent N lanes are determined, they may be determined according to the lane in which the autonomous vehicle is located, the driving intention at that time, and the like. For example, when the vehicle is traveling straight and in the middle lane, the area corresponding to the lane where the vehicle is currently located and the left and right 1 lanes on the navigation path may be determined as the second area. For another example, when the vehicle turns right and is in the rightmost lane, the area corresponding to the lane where the vehicle is currently located and the lane to the left on the navigation path may be determined as the second area. For another example, if the vehicle is traveling straight and is in the leftmost lane, the area corresponding to the lane where the vehicle is currently located and the 1 lanes on the right on the navigation path may be determined as the second area. For another example, when the vehicle turns around and is in the leftmost lane, the area corresponding to the lane where the vehicle is currently located, the lane opposite to the left, and the lane on the right on the navigation path may be determined as the second area.

Still taking fig. 3 as an example, if the navigation path of the automatic driving vehicle a turns right at the intersection, the area corresponding to the lane where the automatic driving vehicle a is currently located and the lane on the left side on the navigation path may be determined as the second area.

After the obstacle information in the first area and the second area is acquired, these obstacles are formed into an obstacle list, and the obstacle list is assumed to include the obstacles b to f. After the scene enumeration, the following scene can be obtained:

scene 1: a, a

Scene 2-6: a+b, a+c, a+d, a+e, a+f

Scenes 7 to 16: a+b+c, a+b+d, a+b+e, a+b+f, a+c+d, a+c+e, a+c+f, a+d+e, a+d+f, a+e+f

It should be noted that, the above scenario enumeration may be combined in order of including the number of entities (the tested autonomous vehicle and the obstacle) from a small number to a large number, and each combination includes at least the tested autonomous vehicle. The number of entities contained in the scene at most can be preset, for example, set to 3, that is, the scene contains at most 3 entities at the time of scene enumeration.

The following describes in detail the step 206, that is, "performing simulation test on the enumerated scenes, respectively, and determining that an abnormal scene with the same abnormal type as the bad case data appears" with reference to the embodiment.

In this step, performing the simulation test refers to restoring the scene, that is, building a simulation environment to restore the position and motion state of the entity included in the scene when the abnormality occurs in the drive test process, so as to check whether the same abnormality still occurs. And respectively carrying out simulation test on each enumerated scene, and if a certain scene has an abnormality of the same type as the bad case data, indicating that the scene is an abnormal scene.

In order to improve the efficiency of the simulation test, the enumerated scenes can be respectively subjected to the simulation test in sequence according to the sequence of the number of entities contained in the scenes from less to more; if the scene of the current simulation test has the same abnormal type as the bad case data, determining the scene of the current simulation test as an abnormal scene, and ending the simulation test on the enumerated scenes respectively; otherwise, continuing to perform simulation test on the next scene.

Taking the scenario enumerated in the above example as an example, first, a simulation test is performed on scenario 1, where scenario 1 only includes the tested autonomous vehicle a. As shown in fig. 4a, the build simulation environment restores the current position, movement state (e.g., speed, attitude, direction) etc. of the tested autonomous vehicle a to check if sudden braking is also occurring (assuming sudden braking occurs here at the time of current road).

And then respectively performing simulation tests on the scenes 2-6. Taking scenario 2 as an example, scenario 2 includes a tested autonomous vehicle a and a pedestrian b. As shown in fig. 4b, the build simulation environment restores the current position, motion state (e.g., speed, attitude, direction) etc. of the tested autonomous vehicle a and pedestrian b to check if sudden braking is also occurring (assuming sudden braking occurs here at the time of current road).

If sudden braking occurs in the scene 2 in the simulation test process of the scenes 2-6, determining the scene 2 as an abnormal scene, and stopping the simulation test on each scene obtained by enumeration; otherwise, continuing the simulation test of the subsequent scene.

In the simulation test of scenes 7 to 16, taking scene 7 as an example, scene 7 includes an automatically driven vehicle a under test, a pedestrian b, and a vehicle c. As shown in fig. 4c, the build simulation environment restores the current positions, motion states (e.g., speed, attitude, direction) and the like of the autonomous vehicle a, pedestrian b and vehicle c under test to verify whether sudden braking is also occurring (assuming sudden braking occurs here at the time of road).

The above step 208, i.e. "using the anomaly scenario to determine the responsible entity of the bad case data from the autonomous vehicle and the obstacle included in the bad case data", is described in detail below in connection with an embodiment.

If the determined abnormal scene only comprises the tested automatic driving vehicle, that is, the scene only comprises the tested automatic driving vehicle when the abnormality occurs in the process of restoring, the problem of the tested automatic driving vehicle is indicated as a responsible entity.

And if the determined abnormal scene comprises the tested automatic driving vehicle and at least one obstacle combination, taking the obstacle contained in the abnormal scene as a responsible entity. For example, scenario 2 is an abnormal scenario, and scenario 2 contains a+b, since scenario 1 has been previously simulation tested to be not an abnormal scenario, b is taken as the responsible entity. For another example, if scene 7 is an abnormal scene, scene 7 contains a+b+c, and it has been simulated that scenes 1, 2 and 3 are not abnormal scenes, and it is stated that b+c together causes an abnormality, so b+c is taken as a responsible entity.

After determining the responsible entity of the bad case data, in order to expand the coverage of the abnormality, the motion information (including the position, the motion state and the like) of the responsible entity in the abnormal scene can be further subjected to generalization processing to obtain a generalization scene, and then the information of the generalization scene is output.

In the generalization process, more instances of the motion information of the responsible entity can be generalized in a sampling form, so that new scenes are formed, and the new scenes are called generalized scenes. Assuming that the scene 2 is an abnormal scene and b is a responsible entity, some new value combinations can be sampled for the information of the position, the speed, the direction and the like of b to obtain new examples as generalization scenes.

And (3) performing simulation test on each obtained generalization scene, and screening to obtain the generalization scene with the same abnormal type as the bad case data. For example, a plurality of generalization scenes are obtained after generalization is performed on the position, the speed, the direction and the like of the b, simulation tests are performed on the generalization scenes, only the generalization scenes which also generate sudden braking of the automatic driving vehicle a are screened out, and information of the generalization scenes is output.

The information of the responsibility entity and the generalization scene can be used for improving an algorithm in the automatic driving vehicle by research and development personnel and operators, and can also be used for forming a test case to carry out targeted test on the automatic driving vehicle.

The foregoing describes specific embodiments of the present disclosure. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims can be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

According to an embodiment of another aspect, a test apparatus is provided. FIG. 5 shows a schematic block diagram of a test apparatus according to one embodiment. As shown in fig. 5, the apparatus 500 may include: the bad case acquisition unit 501, the scene enumeration unit 502, the simulation test unit 503, and the responsibility estimation unit 504 may further include a generalization processing unit 505. Wherein the main functions of each constituent unit are as follows:

the bad case acquisition unit 501 is configured to acquire bad case data of the tested autonomous vehicle.

The scenario enumeration unit 502 is configured to perform scenario enumeration according to the bad case data, where the enumerated scenario includes the tested autonomous vehicle or includes a combination of the tested autonomous vehicle and at least one obstacle included in the bad case data.

The simulation test unit 503 is configured to perform simulation test on the enumerated scenes respectively, and determine that an abnormal scene with the same abnormal type as the bad case data appears.

The responsibility estimating unit 504 is configured to determine a responsibility entity of the bad case data from the tested automatic driving vehicle and the obstacle included in the bad case data by using the abnormal scene.

As one of the realizations, the bad case acquisition unit 501 may acquire the bad case data described above from the data storage device. Wherein the data storage device may be the autonomous vehicle under test, a server or other device having data storage capabilities.

As one of the realizations, the scenario enumeration unit 502 may be specifically configured to: determining an obstacle in a first area and/or a second area according to the bad case data, wherein the first area is an area within a preset first distance range from the position of the tested automatic driving vehicle when the abnormality corresponding to the bad case data occurs, and the second area is a driving area determined according to the navigation path of the tested automatic driving vehicle; taking the tested automatic driving vehicle as one of the scenes, and combining the tested automatic driving vehicle with the determined at least one obstacle respectively to enumerate each scene.

As one of the realizations, the simulation test unit 503 may be specifically configured to: according to the sequence of the number of entities contained in the scenes from less to more, sequentially performing simulation tests on the enumerated scenes, wherein the entities comprise tested automatic driving vehicles or obstacles; if the scene of the current simulation test has the same abnormal type as the bad case data, determining the scene of the current simulation test as an abnormal scene, and ending the simulation test on the enumerated scenes respectively; otherwise, continuing to perform simulation test on the next scene.

The simulation test unit 503 may build a simulation environment to restore the position and motion state of the entity included in the scene when the abnormality occurs in the drive test process, so as to check whether the same abnormality still occurs. And respectively carrying out simulation test on each enumerated scene, and if a certain scene has an abnormality of the same type as the bad case data, indicating that the scene is an abnormal scene.

As one of the realizations, the responsibility estimating unit 504 may be specifically configured to: if the abnormal scene only comprises the tested automatic driving vehicle, determining that the tested automatic driving vehicle is a responsible entity of bad case data; if the abnormal scene comprises a combination of the tested automatic driving vehicle and at least one obstacle, the obstacle contained in the abnormal scene is taken as a responsible entity.

Further, the generalization processing unit 505 is configured to generalize the motion information of the responsible entity in the abnormal scene to obtain a generalized scene; and outputting information of the generalization scene.

As one of the realizable modes, the generalization processing unit 505 can perform simulation test on the generalization scene, and screen to obtain the generalization scene with the same abnormal type as the bad case data; the outputting of the information of the generalized scene includes: and outputting the information of the generalization scene obtained by screening.

The exception types corresponding to the bad case data include: the risk of sudden braking, collision or running exceeds a preset risk level.

In this specification, each embodiment is described in a progressive manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for the device embodiments, since they are substantially similar to the method embodiments, the description is relatively simple, and reference is made to the description of the method embodiments for relevant points. The apparatus embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.

It should be noted that, the user information (including but not limited to user equipment information, user personal information, etc.) and the data (including but not limited to data for analysis, stored data, presented data, etc.) related to the present application are information and data authorized by the user or fully authorized by each party, and the collection, use and processing of the related data need to comply with the related laws and regulations and standards of the related country and region, and provide corresponding operation entries for the user to select authorization or rejection.

In addition, the embodiment of the application further provides a computer readable storage medium, on which a computer program is stored, which when executed by a processor, implements the steps of the method of any one of the foregoing method embodiments.

And an electronic device comprising:

one or more processors; and

a memory associated with the one or more processors for storing program instructions that, when read for execution by the one or more processors, perform the steps of the method of any of the preceding method embodiments.

The present application also provides a computer program product comprising a computer program which, when executed by a processor, implements the steps of the method of any of the preceding method embodiments.

Fig. 6 illustrates an architecture of an electronic device, which may include a processor 610, a video display adapter 611, a disk drive 612, an input/output interface 613, a network interface 614, and a memory 620, to name a few. The processor 610, video display adapter 611, disk drive 612, input/output interface 613, network interface 614, and memory 620 may be communicatively coupled via a communications bus 630.

The processor 610 may be implemented by a general-purpose CPU, a microprocessor, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, etc. for executing relevant programs to implement the technical solutions provided herein.

The Memory 620 may be implemented in the form of ROM (Read Only Memory), RAM (RandomAccess Memory ), a static storage device, a dynamic storage device, or the like. The memory 620 may store an operating system 621 for controlling the operation of the electronic device 600, and a Basic Input Output System (BIOS) 622 for controlling the low-level operation of the electronic device 600. In addition, a web browser 623, a data storage management system 624, a test device 625, and the like may also be stored. The test device 625 may be an application program that specifically implements the operations of the foregoing steps in the embodiments of the present application. In general, when the technical solutions provided in the present application are implemented in software or firmware, relevant program codes are stored in the memory 620 and invoked by the processor 610 to be executed.

The input/output interface 613 is used to connect with an input/output module to realize information input and output. The input/output module may be configured as a component in a device (not shown) or may be external to the device to provide corresponding functionality. Wherein the input devices may include a keyboard, mouse, touch screen, microphone, various types of sensors, etc., and the output devices may include a display, speaker, vibrator, indicator lights, etc.

The network interface 614 is used to connect communication modules (not shown) to enable communication interactions of the device with other devices. The communication module may implement communication through a wired manner (such as USB, network cable, etc.), or may implement communication through a wireless manner (such as mobile network, WIFI, bluetooth, etc.).

Bus 630 includes a path to transfer information between components of the device (e.g., processor 610, video display adapter 611, disk drive 612, input/output interface 613, network interface 614, and memory 620).

It should be noted that although the above devices illustrate only the processor 610, video display adapter 611, disk drive 612, input/output interface 613, network interface 614, memory 620, bus 630, etc., the device may include other components necessary to achieve proper operation in an implementation. Furthermore, it will be understood by those skilled in the art that the above-described apparatus may include only the components necessary to implement the present application, and not all the components shown in the drawings.

From the above description of embodiments, it will be apparent to those skilled in the art that the present application may be implemented in software plus a necessary general purpose hardware platform. Based on such understanding, the technical solutions of the present application may be embodied essentially or in a part contributing to the prior art in the form of a computer program product, which may be stored in a storage medium, such as a ROM/RAM, a magnetic disk, an optical disk, etc., and include several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method described in the embodiments or some parts of the embodiments of the present application.

The foregoing has outlined the detailed description of the preferred embodiment of the present application, and the detailed description of the principles and embodiments of the present application has been provided herein by way of example only to facilitate the understanding of the method and core concepts of the present application; also, as will occur to those of ordinary skill in the art, many modifications are possible in view of the teachings of the present application, both in the detailed description and the scope of its applications. In view of the foregoing, this description should not be construed as limiting the application.

Claims

1. A test method for an autonomous vehicle, the method comprising:

obtaining bad case data of the tested automatic driving vehicle;

2. The method of claim 1, wherein performing scene enumeration based on the bad case data comprises:

3. The method of claim 1, wherein performing the simulation test on the enumerated scenarios respectively comprises:

4. The method of claim 1, wherein determining, with the anomaly scenario, responsible entities for the bad case data from the autonomous vehicle and obstacles contained in the bad case data comprises:

5. The method according to claim 1, wherein the method further comprises:

and outputting the information of the generalization scene.

6. The method of claim 5, further comprising, prior to outputting the information of the generalized scene:

7. The method according to any one of claims 1 to 6, wherein the anomaly type corresponding to the bad case data includes: the risk of sudden braking, collision or running exceeds a preset risk level.

8. A test device for an autonomous vehicle, the device comprising:

9. A computer readable storage medium, on which a computer program is stored, characterized in that the program, when being executed by a processor, implements the steps of the method according to any one of claims 1 to 7.

10. An electronic device, comprising:

one or more processors; and

a memory associated with the one or more processors for storing program instructions that, when read for execution by the one or more processors, perform the steps of the method of any of claims 1 to 7.