CN111765903B

CN111765903B - Test method, device, electronic device and medium for automatic driving vehicle

Info

Publication number: CN111765903B
Application number: CN202010606636.6A
Authority: CN
Inventors: 赵军
Original assignee: Apollo Intelligent Technology Beijing Co Ltd
Current assignee: Apollo Intelligent Technology Beijing Co Ltd
Priority date: 2020-06-29
Filing date: 2020-06-29
Publication date: 2022-08-09
Anticipated expiration: 2040-06-29
Also published as: EP3822608A1; JP2022013646A; KR102498441B1; US20210403011A1; CN111765903A; JP7112544B2; KR20220001431A

Abstract

The present disclosure provides a test method for an automatic driving vehicle, which relates to the field of automatic driving, and comprises the following steps: the method comprises the steps of obtaining test data generated in the test process of a test site, wherein the test data comprises a corresponding relation between the current accumulated problem quantity monitored in the test process and the current driving mileage of a vehicle, determining a corresponding relation between a problem monitoring proportion and the current driving mileage based on the test data, wherein the problem monitoring proportion comprises a proportion between the monitored current accumulated problem quantity and the total quantity of problems monitored in the test process, and fitting a preset evaluation model to obtain an optimized evaluation model based on the corresponding relation between the problem monitoring proportion and the current driving mileage, wherein the optimized evaluation model is used for evaluating the corresponding relation between the problem monitoring proportion and the test mileage of the test site in each test process.

Description

Test method, device, electronic device and medium for automatic driving vehicle

Technical Field

The present disclosure relates to the field of autonomous driving, and more particularly, to a method and apparatus for testing an autonomous vehicle, an electronic device, and a computer-readable medium.

Background

With the rapid development of vehicle technology and electronic technology, autonomous vehicles are increasingly emerging in people's lives. The automatic driving vehicle can obtain the information of the traffic scene of the vehicle through various sensors, and determine a proper automatic driving strategy according to the traffic scene information so as to realize the automatic driving of the vehicle.

During the development of autonomous vehicles, road running tests are typically required for autonomous vehicles to test various devices and programs in the autonomous vehicles.

However, in implementing the concept of the present disclosure, the inventors found that at least the following problems exist in the related art: in order to fully test the automatic driving vehicles, a large number of automatic driving vehicles are usually adopted to perform repeated tests in the same test field to ensure the test coverage rate, so that the test cost is high and the test efficiency is low.

Disclosure of Invention

In view of the above, the present disclosure provides a test method, apparatus, electronic device and computer readable medium for an autonomous vehicle.

One aspect of the present disclosure provides a test method of an autonomous vehicle, including: the method comprises the steps of obtaining test data generated in a test process of a test site, wherein the test data comprises a corresponding relation between a current accumulated problem quantity monitored in the test process and the current driving mileage of a vehicle, determining a corresponding relation between a problem monitoring proportion and the current driving mileage based on the test data, wherein the problem monitoring proportion comprises a proportion between the monitored current accumulated problem quantity and the total quantity of problems monitored in the test process, and fitting a preset evaluation model to obtain an optimized evaluation model based on the corresponding relation between the problem monitoring proportion and the current driving mileage, wherein the optimized evaluation model is used for evaluating the corresponding relation between the problem monitoring proportion and the test mileage of the test site in each test process.

According to an embodiment of the present disclosure, the method further comprises: and aiming at the expected problem monitoring proportion, determining a test mileage to be tested by utilizing the optimized evaluation model, and executing the test of the automatic driving vehicle based on the test mileage to be tested.

According to an embodiment of the present disclosure, the test data further comprises: an issue record of issues monitored during the testing process, the issue record including at least one issue description, each issue description of the at least one issue description including at least one description label.

According to an embodiment of the present disclosure, the method further comprises: recording test data generated during the test with respect to the test site. The recording test data generated during the test with respect to the test site includes: responding to the monitoring of the new problems in the test process, determining whether other problems with the same problem records as the new problems exist in the monitored problems, if not, recording the problem records of the new problems, updating the current accumulated problem quantity, and recording the corresponding relation between the current accumulated problem quantity and the current driving mileage.

According to an embodiment of the present disclosure, the recording of the correspondence between the current accumulated problem quantity and the current mileage includes: responding to the update of the current accumulated problem quantity, acquiring the updated current accumulated problem quantity and the current driving mileage, and recording the corresponding relation between the updated current accumulated problem quantity and the current driving mileage, or periodically acquiring the current accumulated problem quantity and the current driving mileage, and recording the corresponding relation between the current accumulated problem quantity and the current driving mileage.

According to an embodiment of the present disclosure, the preset evaluation model includes an exponential model.

According to an embodiment of the present disclosure, the preset evaluation model is represented as:

y＝1-n ^x

wherein y represents a problem monitoring proportion, x represents the current driving mileage, and n is a parameter of the preset model.

According to an embodiment of the present disclosure, the problem record includes at least one problem description of a static scene description, a dynamic interaction behavior description, and an unreasonable behavior description.

According to an embodiment of the present disclosure, the static scene description includes at least one description label in intersection left turn, intersection right turn, intersection straight going, intersection turning around, non-intersection driving, roundabout, overpass, branch road, merging area, main and auxiliary road, ramp and temporary road construction. The dynamic interaction description comprises at least one description tag of none, vehicle, pedestrian, non-motor vehicle and other obstacles. The dynamic interactive behavior description comprises at least one description label in the processes of no, side-by-side, car following, lane changing, car cutting, side parking and starting. The unreasonable behavior description comprises at least one descriptive label of unreasonable braking, accident-free braking, unreasonable acceleration, too fast speed, too slow speed, left-right swinging, transverse deviation, position drift, transverse too close, positioning error, recognition error, violation of contra-ordination, redundant behavior and opportunity misalignment.

Another aspect of the disclosure provides a testing apparatus of an autonomous vehicle, the apparatus including an acquisition module, a first determination module, and a fitting processing module. The acquisition module is used for acquiring test data generated in a test process about a test site, wherein the test data comprises a corresponding relation between the current accumulated problem quantity monitored in the test process and the current driving mileage of the vehicle. The first determining module is used for determining a corresponding relation between a problem monitoring proportion and the current driving mileage based on the test data, wherein the problem monitoring proportion comprises the proportion between the monitored current accumulated problem quantity and the monitored total problem quantity in the test process. The fitting processing module is used for fitting a preset evaluation model to obtain an optimized evaluation model based on the corresponding relation between the problem monitoring proportion and the current driving mileage, wherein the optimized evaluation model is used for evaluating the corresponding relation between the problem monitoring proportion and the testing mileage of the testing field in each testing process.

Another aspect of the present disclosure provides an electronic device including: one or more processors, a storage device to store one or more programs, wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to perform the method as described above.

Another aspect of the present disclosure provides a computer-readable storage medium storing computer-executable instructions for implementing the method as described above when executed.

Another aspect of the disclosure provides a computer program comprising computer executable instructions for implementing the method as described above when executed.

Drawings

The above and other objects, features and advantages of the present disclosure will become more apparent from the following description of embodiments of the present disclosure with reference to the accompanying drawings, in which:

FIG. 1 schematically illustrates a system architecture for a test method of an autonomous vehicle according to an embodiment of the disclosure;

FIG. 2 schematically illustrates a flow chart of a test method of an autonomous vehicle according to an embodiment of the disclosure;

FIG. 3 schematically shows a schematic diagram of an assessment model according to an embodiment of the present disclosure;

FIG. 4 schematically illustrates a block diagram of a testing arrangement for an autonomous vehicle according to an embodiment of the disclosure; and

FIG. 5 schematically shows a block diagram of an electronic device according to an embodiment of the disclosure.

Detailed Description

Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. It should be understood that these descriptions are illustrative only and are not intended to limit the scope of the present disclosure. In the following detailed description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the embodiments of the disclosure. It may be evident, however, that one or more embodiments may be practiced without these specific details. Moreover, in the following description, descriptions of well-known structures and techniques are omitted so as to not unnecessarily obscure the concepts of the present disclosure.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. The terms "comprises," "comprising," and the like, as used herein, specify the presence of stated features, steps, operations, and/or components, but do not preclude the presence or addition of one or more other features, steps, operations, or components.

All terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art unless otherwise defined. It is noted that the terms used herein should be interpreted as having a meaning that is consistent with the context of this specification and should not be interpreted in an idealized or overly formal sense.

Where a convention analogous to "at least one of A, B and C, etc." is used, in general such a construction is intended in the sense one having skill in the art would understand the convention (e.g., "a system having at least one of A, B and C" would include but not be limited to systems that have a alone, B alone, C alone, a and B together, a and C together, B and C together, and/or A, B, C together, etc.). Where a convention analogous to "A, B or at least one of C, etc." is used, in general such a construction is intended in the sense one having skill in the art would understand the convention (e.g., "a system having at least one of A, B or C" would include but not be limited to systems that have a alone, B alone, C alone, a and B together, a and C together, B and C together, and/or A, B, C together, etc.).

The present disclosure provides a test method of an autonomous vehicle, the method comprising: and acquiring test data generated in the test process about the test site, wherein the test data comprises the corresponding relation between the current accumulated problem quantity monitored in the test process and the current driving mileage of the vehicle. Then, based on the test data, determining a corresponding relation between a problem monitoring proportion and the current driving mileage, wherein the problem monitoring proportion comprises a proportion between the monitored current accumulated problem quantity and the monitored total problem quantity in the test process. And fitting the preset evaluation model based on the corresponding relation between the problem monitoring proportion and the current driving mileage to obtain an optimized evaluation model, wherein the optimized evaluation model is used for evaluating the corresponding relation between the problem monitoring proportion and the testing mileage of the testing field in each testing process.

Fig. 1 schematically illustrates a system architecture 100 of a test method of an autonomous vehicle according to an embodiment of the disclosure.

It should be noted that fig. 1 is only an example of a system architecture to which the embodiments of the present disclosure may be applied to help those skilled in the art understand the technical content of the present disclosure, and does not mean that the embodiments of the present disclosure may not be applied to other devices, systems, environments or scenarios.

As shown in fig. 1, a system architecture 100 according to this embodiment may include an autonomous vehicle 101, a network 102, and a server 103. Network 102 is the medium used to provide a communication link between autonomous vehicle 101 and server 103. Network 102 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.

According to an embodiment of the present disclosure, autonomous vehicle 101 may be, for example, a smart car that is unmanned via a computer system. Autonomous vehicle 101 may, for example, integrate environmental awareness and planning decision making. For example, the autonomous vehicle 101 may be equipped with a radar sensor or a monitoring device, etc., to sense and monitor the surrounding environment and traffic conditions.

According to an embodiment of the present disclosure, the server 103 may be a server that provides various services. For example, the server 103 may acquire test data generated by the autonomous vehicle 101 during a test at a test site, and perform fitting processing on a preset evaluation model based on the test data.

It should be noted that the test method for the autonomous vehicle provided by the embodiment of the present disclosure may be generally executed by the server 103. Accordingly, the testing apparatus for an autonomous vehicle provided by the embodiments of the present disclosure may be generally disposed in the server 103.

For example, the server 103 may obtain test data generated during a full test of a plurality of autonomous vehicles 101 with respect to a test site. And fitting the preset evaluation model based on the test data to obtain an optimized evaluation model, wherein the optimized evaluation model can be used for evaluating the corresponding relation between the problem monitoring proportion and the test mileage of the test site in each test process.

It can be appreciated that it is difficult to determine when to end the test during the road testing phase of an autonomous vehicle. When a huge automatic driving system is tested in a road network which cannot be described regularly, all problems need to be found at great cost. When the problem monitoring proportion is far from disproportionate with the testing mileage, the testing efficiency can be reduced and manpower and material resources are wasted by continuously testing.

In view of this, the optimized evaluation model obtained in the embodiment of the present disclosure may be used to evaluate a correspondence between a problem monitoring ratio and a test mileage of a relevant test site in each test process, so that the validity of the drive test may be quantitatively evaluated. Therefore, a proper test mileage can be determined based on the optimized evaluation model of the embodiment of the disclosure, and the test can be finished after the test mileage is completed, so that the test sufficiency is ensured, the test efficiency can be improved, and the test cost can be saved.

It should be understood that the number of autonomous vehicles and servers in fig. 1 is merely illustrative. There may be any number of autonomous vehicles and servers, as desired.

FIG. 2 schematically shows a flow chart of a testing method of an autonomous vehicle according to an embodiment of the disclosure.

As shown in fig. 2, the method includes operations S201 to S203.

In operation S201, test data generated during a test with respect to a test site is acquired, the test data including a correspondence between a current cumulative number of problems monitored during the test and a current mileage of the vehicle.

According to an embodiment of the present disclosure, a test procedure for a test site may be, for example, a full test procedure for the test site. For example, test data generated during a test in which the test mileage with respect to the test site is greater than a first threshold and/or the number of monitored problems is greater than a second threshold may be acquired, so that more sophisticated data generated during a sufficient test with respect to the test site may be acquired.

According to the embodiment of the disclosure, the generated test data can be recorded in the test process, so that the test data can be processed later.

For example, during the testing process, each test vehicle may monitor itself for problems and, if a problem occurs, generate a problem record for the problem. For example, it may be monitored whether an autonomous vehicle is experiencing unreasonable driving behavior, and if so, a problem record may be generated regarding the unreasonable driving behavior.

In an embodiment of the present disclosure, the issue record may include at least one issue description, and each issue description of the at least one issue description may include at least one description tag. For example, the issue record may include at least one of a static scene description, a dynamic interaction behavior description, and an unreasonable driving behavior description at the time of the issue occurrence. Wherein the static scene description may include at least one description label in the construction of crossroad left turn, crossroad right turn, crossroad straight run, crossroad turn around, non-crossroad running, roundabout, overpass, branch road, merging area, main and auxiliary roads, ramp and temporary road. The dynamic interaction description may include at least one description tag of none, vehicle, pedestrian, non-motor vehicle, and other obstruction. The dynamic interactive behavior description may include at least one description tag of none, side-by-side, following, lane change, cutting, parking beside, and take off. Unreasonable behavior descriptions may include at least one descriptive label of unreasonable braking, casual braking, unreasonable acceleration, excessive speed, yaw, lateral offset, positional drift, lateral proximity, positioning error, recognition error, violation of contra-ordination, redundant behavior, and timing misalignment.

The problem description required to be contained by the problem record can be normalized, and the description label is preset for each problem description, so that the generated problem record has normalized content, and whether the problems are the same or not can be determined through the problem record. For example, the monitored problem record for problem 1 can be { intersection left turn, vehicle, side-by-side, laterally too close }, the problem record for problem 2 can be { intersection left turn, vehicle, following, redundant behavior }, the problem record for problem 3 can be { intersection left turn, vehicle, side-by-side, laterally too close }, then problem 3 is the same problem as problem 1, and problem 2 is a different problem than problem 1.

In the embodiment of the disclosure, in response to monitoring a new problem in a test process, determining whether other problems having the same problem record as that of the new problem exist in the monitored problems, if not, recording the problem record of the new problem, updating the current accumulated problem quantity, and recording the corresponding relationship between the current accumulated problem quantity and the current mileage.

For example, a problem record may be generated for an autonomous vehicle in response to monitoring that a problem occurred with the vehicle. And then judging whether the same problem exists in the monitored problems before based on the problem record, if so, indicating that the problem is found, not recording and processing the problem, if not, indicating that the problem is a new problem which is not found, recording the problem record of the new problem, and updating the current accumulated problem quantity.

In an embodiment of the present disclosure, in response to the update of the current accumulated question quantity, the updated current accumulated question quantity and the current mileage are obtained, and the corresponding relationship between the updated current accumulated question quantity and the current mileage is recorded.

For example, at the first time during the test, problem 1 is monitored, and the problem record for problem 1 may be recorded and the number of currently accumulated problems may be updated to 1. Obtaining the current driving mileage S corresponding to the first time ₁ And recording the current accumulated problem quantity 1 and the current driving mileage S ₁ The corresponding relation between them. And monitoring the problem 2 at a second time after the first time, determining whether the problem 2 is the same as the problem 1 or not through the problem records of the problem 1 and the problem 2, if so, not processing the problem, and if not, recording the problem record of the problem 2 and updating the number of the current accumulated problems to be 2. Obtaining the current driving mileage S corresponding to the second time ₂ And recording the current accumulated problem number 2 and the current driving mileage S ₂ The corresponding relation between them. And monitoring the problem 3 at a third time after the second time, determining whether the problem 3 is the same as the problem 1 or the problem 2 through the problem records of the problem 1, the problem 2 and the problem 3, if the problem 3 is the same as the problem 1 or the problem 2, not processing the problem, and if the problem 3 is not the same as the problem 2, recording the problem record of the problem 3 and updating the number of the current accumulated problems to 3. Obtaining the current driving mileage S corresponding to the third time ₃ And recording the current accumulated problem number 3 and the current driving mileage S ₃ The corresponding relationship between the two groups, and so on. It can be understood that when a plurality of test vehicles participate in the test process, the current problem accumulation number is the accumulation number of different problems monitored by all the test vehicles participating in the test, and the current driving mileage is the sum of the driving mileage of all the test vehicles participating in the test.

In another embodiment of the present disclosure, the current accumulated problem quantity and the current mileage may also be periodically obtained, and the corresponding relationship between the current accumulated problem quantity and the current mileage is recorded.

For example, the period may be one day, and after the test on the first day is completed, the sum S of the records of all the problems monitored by each test vehicle on the first day and the mileage traveled by each test vehicle on the same day may be obtained ₁ . Determining the number of non-identical problems as the current cumulative number of problems X based on the problem record ₁ Record X ₁ And S ₁ The corresponding relation between them. After the test on the next day is finished, the sum S of the records of all problems monitored by each test vehicle on the next day and the mileage of each test vehicle in the current day can be obtained ₂ . Determining the number of different problems in all the problems monitored on the first day and the second day as the current accumulated problem number X based on the problem records ₂ Record X ₂ And (S) ₁ +S ₂ ) The corresponding relation between them.

It is understood that the present disclosure does not limit the recording manner of the corresponding relationship between the current accumulated problem quantity and the current mileage, and those skilled in the art can set the relationship according to actual situations. For example, the method of the previous embodiment of the disclosure has higher granularity, can obtain more data, and improves the fitting accuracy, and the method of the later embodiment of the disclosure has lower granularity, can save the calculation resources, and improves the calculation efficiency.

According to the embodiment of the disclosure, the server can acquire the problem records and the mileage on the same day recorded by each test vehicle after the test on each day is finished, so as to determine the corresponding relation between the current accumulated problem quantity and the current mileage. Alternatively, the server may also respond to a test vehicle monitoring that a problem occurs in itself, and obtain a problem record of the problem and the current driving mileage of all vehicles, so as to determine the corresponding relationship between the current accumulated problem and the current driving mileage. The method is not limited by the disclosure, and a person skilled in the art can set the method according to actual conditions, and the method only needs to acquire the corresponding relationship between the number of the current cumulative problems and the current mileage so as to fit the model.

In operation S202, a corresponding relationship between a problem monitoring ratio and a current driving distance is determined based on the test data, wherein the problem monitoring ratio includes a ratio between a monitored current accumulated problem quantity and a total quantity of problems monitored in the test process.

According to the embodiment of the disclosure, the total number X of all the different problems monitored in the test process and the current accumulated problem number X can be obtained _i With the current driving distance S _i The corresponding relation between them. So that it can be based on the total number of problems X and the respective current cumulative number of problems X _i With the current driving distance S _i The corresponding relationship between the problem monitoring proportion and the current driving mileage is determined. For example, as shown in table 1, where X is the total number of problems monitored during this test, and S is the total driving range during this test.

Current cumulative number of problems	Current mileage	Problem monitoring ratio
			X ₁	S ₁	X ₁ /X
X ₂	S ₂	X ₂ /X
			…	…	…
X	S	X/X

TABLE 1 corresponding relationship between problem monitoring ratio and current mileage

In operation S203, a preset evaluation model is fitted to obtain an optimized evaluation model based on a corresponding relationship between the problem monitoring ratio and the current driving mileage, where the optimized evaluation model is used to evaluate a corresponding relationship between the problem monitoring ratio and the testing mileage of the test site in each test process.

According to an embodiment of the present disclosure, the preset evaluation model includes an exponential model. For example, the preset evaluation model may be expressed as:

y＝1-n ^x

wherein y represents the problem monitoring proportion, x represents the testing mileage, and n is the parameter of the preset model.

In the embodiment of the present disclosure, the corresponding relationship between each problem monitoring proportion and the current driving mileage can be obtained by using the above steps, and the preset evaluation model is fitted to obtain the value of the model parameter n belonging to the test site, so that the optimized evaluation model related to the test site can be obtained.

According to the embodiment of the disclosure, the parameters n in the preset evaluation model fitted based on the test data generated in the test process in different fields are different. That is, the optimized evaluation model obtained in the embodiment of the present disclosure may be used to evaluate the correspondence between the problem monitoring proportion and the test mileage of the test site corresponding to the optimized evaluation model in each test process.

For example, fig. 3 schematically shows a schematic diagram of an evaluation model according to an embodiment of the present disclosure. As shown in fig. 3, the abscissa x of the evaluation model obtained by fitting represents the test mileage, and the ordinate y represents the problem monitoring ratio.

In the disclosed embodiment, during the subsequent repeated debugging and testing process on the test site, the fitted evaluation model can be used to determine the mileage required to be tested, and after the driving mileage reaches the mileage, the test can be completed.

For example, a proportion may be monitored for an expected problem, a test mileage to be tested is determined using an optimization evaluation model, and a test of the autonomous vehicle is performed based on the test mileage to be tested.

For example, in the model shown in FIG. 3, the slope of the curve becomes lower as x increases. That is, in the later testing process, a problem can be found only by running a long testing distance, and the testing efficiency is obviously reduced.

Thus, in embodiments of the present disclosure, a relatively suitable problem monitoring proportion (e.g., 80%) may be selected and a test mileage over which 80% of the problems need to be completed is predicted based on the fitted evaluation model, so that it may be determined when to terminate the test based on the test mileage during the next test.

The embodiment of the disclosure can fit the evaluation model through the historical test data about a certain test site, so that an optimized evaluation model about the test site can be obtained, the evaluation model can quantitatively evaluate the test site, and a relationship between a test mileage (i.e., a test quantity) and a problem monitoring proportion (i.e., test sufficiency) can be prepared, so as to improve the test efficiency and reduce the test cost.

The problem records of all problems are recorded in a standardized mode, so that whether newly found problems are found before can be determined through the problem records, efficiency can be improved, and communication cost can be reduced.

Fig. 4 schematically shows a block diagram of a test apparatus 400 of an autonomous vehicle according to an embodiment of the disclosure.

As shown in fig. 4, the apparatus 400 includes an acquisition module 410, a first determination module 420, and a fitting processing module 430.

The obtaining module 410 is configured to obtain test data generated in a test process about a test site, where the test data includes a corresponding relationship between a current accumulated number of problems monitored in the test process and a current mileage of the vehicle. According to the embodiment of the present disclosure, the obtaining module 410 may, for example, perform the operation S201 described above with reference to fig. 2, which is not described herein again.

The first determining module 420 is configured to determine a corresponding relationship between a problem monitoring ratio and a current driving distance based on the test data, where the problem monitoring ratio includes a ratio between a monitored current accumulated problem quantity and a monitored total problem quantity in the test process. According to the embodiment of the present disclosure, the first determining module 420 may perform, for example, the operation S202 described above with reference to fig. 2, which is not described herein again.

The fitting processing module 430 is configured to perform fitting processing on the preset evaluation model based on a corresponding relationship between the problem monitoring proportion and the test mileage to obtain an optimized evaluation model, where the optimized evaluation model is configured to evaluate a corresponding relationship between the problem monitoring proportion and the test mileage in each test process of the test site. According to the embodiment of the present disclosure, the fitting processing module 430 may, for example, perform operation S203 described above with reference to fig. 2, which is not described herein again.

According to an embodiment of the present disclosure, the apparatus 400 may further include a second determining module and a test executing module (not shown in the figure). The second determining module is used for determining the testing mileage to be tested by utilizing the optimization evaluation model aiming at the expected problem monitoring proportion. The test execution module is used for executing the test of the automatic driving vehicle based on the test mileage to be tested.

According to an embodiment of the present disclosure, the apparatus 400 further comprises: and a recording module (not shown) for recording test data generated during the test with respect to the test site. Recording test data generated during the test with respect to the test site, comprising: and responding to the monitoring of the new problems in the test process, determining whether other problems with the same problems as the problem records of the new problems exist in the monitored problems, if not, recording the problem records of the new problems, updating the current accumulated problem quantity, and recording the corresponding relation between the current accumulated problem quantity and the current driving mileage.

According to the embodiment of the disclosure, recording the corresponding relation between the current accumulated problem quantity and the current driving mileage includes: and responding to the update of the current accumulated problem quantity, acquiring the updated current accumulated problem quantity and the current driving mileage, and recording the corresponding relation between the updated current accumulated problem quantity and the current driving mileage, or periodically acquiring the current accumulated problem quantity and the current driving mileage, and recording the corresponding relation between the current accumulated problem quantity and the current driving mileage.

y＝1-n ^x

wherein y represents the problem monitoring proportion, x represents the current driving mileage, and n is a parameter of the preset model.

According to an embodiment of the present disclosure, the issue record includes at least one of a static scenario description, a dynamic interaction behavior description, and an unreasonable behavior description.

According to an embodiment of the present disclosure, the static scene description includes at least one description label in intersection left turn, intersection right turn, intersection straight run, intersection turning around, non-intersection driving, roundabout, overpass, branch road, merging area, main and auxiliary road, ramp and temporary road construction. The dynamic interaction description comprises at least one description tag of none, vehicle, pedestrian, non-motor vehicle and other obstacles. The dynamic interactive behavior description comprises at least one description label of none, side-by-side, car following, lane changing, car cutting, side parking and starting. The unreasonable behavior description comprises at least one descriptive label of unreasonable braking, accident-free braking, unreasonable acceleration, too fast speed, too slow speed, left and right swinging, transverse deviation, position drift, transverse too close, positioning error, recognition error, violation of contra-ordination, redundant behavior and opportunity misalignment.

Any number of modules, sub-modules, units, sub-units, or at least part of the functionality of any number thereof according to embodiments of the present disclosure may be implemented in one module. Any one or more of the modules, sub-modules, units, sub-units according to the embodiments of the present disclosure may be implemented by being split into a plurality of modules. Any one or more of the modules, sub-modules, units, sub-units according to embodiments of the present disclosure may be implemented at least in part as a hardware circuit, such as a Field Programmable Gate Array (FPGA), a Programmable Logic Array (PLA), a system on a chip, a system on a substrate, a system on a package, an Application Specific Integrated Circuit (ASIC), or may be implemented in any other reasonable manner of hardware or firmware by integrating or packaging a circuit, or in any one of or a suitable combination of software, hardware, and firmware implementations. Alternatively, one or more of the modules, sub-modules, units, sub-units according to embodiments of the disclosure may be at least partially implemented as a computer program module, which when executed may perform the corresponding functions.

For example, any of the obtaining module 410, the first determining module 420, and the fitting processing module 430 may be combined and implemented in one module, or any one of them may be split into a plurality of modules. Alternatively, at least part of the functionality of one or more of these modules may be combined with at least part of the functionality of the other modules and implemented in one module. According to an embodiment of the present disclosure, at least one of the obtaining module 410, the first determining module 420, and the fitting processing module 430 may be implemented at least in part as a hardware circuit, such as a Field Programmable Gate Array (FPGA), a Programmable Logic Array (PLA), a system on a chip, a system on a substrate, a system on a package, an Application Specific Integrated Circuit (ASIC), or may be implemented in hardware or firmware in any other reasonable manner of integrating or packaging a circuit, or in any one of or a suitable combination of software, hardware, and firmware. Alternatively, at least one of the obtaining module 410, the first determining module 420 and the fitting processing module 430 may be at least partially implemented as a computer program module, which when executed may perform the respective functions.

Fig. 5 schematically shows a block diagram of an electronic device adapted to implement the above described method according to an embodiment of the present disclosure. The electronic device shown in fig. 5 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present disclosure.

As shown in fig. 5, an electronic device 500 according to an embodiment of the present disclosure includes a processor 501 that can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM)502 or a program loaded from a storage section 508 into a Random Access Memory (RAM) 503. The processor 501 may comprise, for example, a general purpose microprocessor (e.g., a CPU), an instruction set processor and/or associated chipset, and/or a special purpose microprocessor (e.g., an Application Specific Integrated Circuit (ASIC)), among others. The processor 501 may also include onboard memory for caching purposes. Processor 501 may include a single processing unit or multiple processing units for performing different actions of a method flow according to embodiments of the disclosure.

In the RAM 503, various programs and data necessary for the operation of the electronic apparatus 500 are stored. The processor 501, the ROM502, and the RAM 503 are connected to each other by a bus 504. The processor 501 performs various operations of the method flows according to the embodiments of the present disclosure by executing programs in the ROM502 and/or the RAM 503. Note that the programs may also be stored in one or more memories other than the ROM502 and the RAM 503. The processor 501 may also perform various operations of method flows according to embodiments of the present disclosure by executing programs stored in the one or more memories.

According to an embodiment of the present disclosure, electronic device 500 may also include an input/output (I/O) interface 505, input/output (I/O) interface 505 also being connected to bus 504. The electronic device 500 may also include one or more of the following components connected to the I/O interface 505: an input portion 506 including a keyboard, a mouse, and the like; an output portion 507 including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage portion 508 including a hard disk and the like; and a communication section 509 including a network interface card such as a LAN card, a modem, or the like. The communication section 509 performs communication processing via a network such as the internet. The driver 510 is also connected to the I/O interface 505 as necessary. A removable medium 511 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 510 as necessary, so that a computer program read out therefrom is mounted into the storage section 508 as necessary.

According to embodiments of the present disclosure, method flows according to embodiments of the present disclosure may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable storage medium, the computer program containing program code for performing the method illustrated by the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication section 509, and/or installed from the removable medium 511. The computer program, when executed by the processor 501, performs the above-described functions defined in the system of the embodiments of the present disclosure. The systems, devices, apparatuses, modules, units, etc. described above may be implemented by computer program modules according to embodiments of the present disclosure.

The present disclosure also provides a computer-readable storage medium, which may be embodied in the device/apparatus/system described in the above embodiments; or may exist separately and not be assembled into the device/apparatus/system. The computer-readable storage medium carries one or more programs which, when executed, implement the method according to an embodiment of the disclosure.

According to embodiments of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, which may include, for example but is not limited to: a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. For example, according to embodiments of the present disclosure, a computer-readable storage medium may include ROM502 and/or RAM 503 and/or one or more memories other than ROM502 and RAM 503 described above.

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

Those skilled in the art will appreciate that various combinations and/or combinations of features recited in the various embodiments and/or claims of the present disclosure can be made, even if such combinations or combinations are not expressly recited in the present disclosure. In particular, various combinations and/or combinations of the features recited in the various embodiments and/or claims of the present disclosure may be made without departing from the spirit or teaching of the present disclosure. All such combinations and/or associations are within the scope of the present disclosure.

The embodiments of the present disclosure have been described above. However, these examples are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Although the embodiments are described separately above, this does not mean that the measures in the embodiments cannot be used in advantageous combination. The scope of the disclosure is defined by the appended claims and equivalents thereof. Various alternatives and modifications can be devised by those skilled in the art without departing from the scope of the present disclosure, and such alternatives and modifications are intended to be within the scope of the present disclosure.

Claims

1. A method of testing an autonomous vehicle, comprising:

acquiring test data generated in a test process about a test site, wherein the test data comprises a corresponding relation between a current accumulated problem quantity monitored in the test process and a current driving mileage of the vehicle;

determining a corresponding relation between a problem monitoring proportion and the current driving mileage based on the test data, wherein the problem monitoring proportion comprises the proportion between the monitored current accumulated problem quantity and the total quantity of the problems monitored in the test process; and

fitting a preset evaluation model to obtain an optimized evaluation model based on the corresponding relation between the problem monitoring proportion and the current driving mileage, wherein the optimized evaluation model is used for evaluating the corresponding relation between the problem monitoring proportion and the testing mileage of the testing field in each testing process,

wherein the preset evaluation model comprises an exponential model.

2. The method of claim 1, further comprising:

aiming at the expected problem monitoring proportion, determining the testing mileage to be tested by utilizing the optimized evaluation model; and

performing a test of the autonomous vehicle based on the test mileage to be tested.

3. The method of claim 1, wherein the test data further comprises:

an issue record of issues monitored during the testing process, the issue record including at least one issue description, each issue description of the at least one issue description including at least one description label.

4. The method of claim 3, further comprising:

recording test data generated during the test procedure with respect to the test site;

wherein the recording test data generated during the test with respect to the test site comprises:

in response to monitoring a new issue during the testing process, determining whether there are already other issues in the monitored issues that have the same issue record as the new issue;

if not, recording the problem record of the new problem, and updating the current accumulated problem quantity; and

and recording the corresponding relation between the current accumulated problem quantity and the current driving mileage.

5. The method of claim 4, wherein said recording a correspondence between a current cumulative number of questions and a current mileage traveled comprises:

responding to the update of the current accumulated problem quantity, acquiring the updated current accumulated problem quantity and the current driving mileage, and recording the corresponding relation between the updated current accumulated problem quantity and the current driving mileage; or

Periodically acquiring the current accumulated problem quantity and the current driving mileage, and recording the corresponding relation between the current accumulated problem quantity and the current driving mileage.

6. The method according to claim 1, wherein the preset evaluation model is represented as:

y＝1-n ^x

7. The method of claim 3, wherein the issue record includes at least one of a static scenario description, a dynamic interaction behavior description, and an unreasonable behavior description.

8. The method of claim 7, wherein,

the static scene description comprises at least one description label in the construction of crossroad left turn, crossroad right turn, crossroad straight running, crossroad turn around, non-crossroad running, roundabout, overpass, branch road, merging area, main and auxiliary roads, ramp and temporary road;

the dynamic interaction description comprises at least one description label of none, vehicle, pedestrian, non-motor vehicle and other obstacles;

the dynamic interactive behavior description comprises at least one description label in the processes of no, side-by-side, car following, lane changing, car cutting, side parking and starting;

the unreasonable behavior description comprises at least one descriptive label of unreasonable braking, accident-free braking, unreasonable acceleration, too fast speed, too slow speed, left-right swinging, transverse deviation, position drift, transverse too close, positioning error, recognition error, violation of contra-ordination, redundant behavior and opportunity misalignment.

9. A test apparatus for an autonomous vehicle, comprising:

the system comprises an acquisition module, a display module and a control module, wherein the acquisition module is used for acquiring test data which is generated in a test process and related to a test site, and the test data comprises a corresponding relation between the current accumulated problem quantity monitored in the test process and the current driving mileage of the vehicle;

the first determination module is used for determining the corresponding relation between the problem monitoring proportion and the current driving mileage based on the test data, wherein the problem monitoring proportion comprises the proportion between the monitored current accumulated problem quantity and the monitored total problem quantity in the test process; and

a fitting processing module for fitting a preset evaluation model based on the corresponding relationship between the problem monitoring proportion and the current driving mileage to obtain an optimized evaluation model, wherein the optimized evaluation model is used for evaluating the corresponding relationship between the problem monitoring proportion and the testing mileage of the test site in each test process,

wherein the preset evaluation model comprises an exponential model.

10. The apparatus of claim 9, further comprising:

the second determination module is used for determining the testing mileage to be tested by utilizing the optimized evaluation model aiming at the expected problem monitoring proportion; and

a test execution module for executing a test of the autonomous vehicle based on the test mileage to be tested.

11. The apparatus of claim 9, wherein the test data further comprises:

12. The apparatus of claim 11, further comprising:

the recording module is used for recording test data generated in the test process about the test site;

13. The apparatus of claim 12, wherein said recording a correspondence between a current cumulative number of questions and a current mileage traveled comprises:

14. The apparatus of claim 9, wherein the preset evaluation model is represented as:

y＝1-n ^x

15. The apparatus of claim 11, wherein the issue record includes at least one of a static scenario description, a dynamic interaction behavior description, and an unreasonable behavior description.

16. The apparatus of claim 15, wherein,

17. An electronic device, comprising:

one or more processors;

a storage device for storing one or more programs,

wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to perform the method of any of claims 1-8.

18. A computer readable medium having stored thereon executable instructions which, when executed by a processor, cause the processor to perform the method of any one of claims 1 to 8.