CN112818845A

CN112818845A - Test method, target object detection method, driving control method and device

Info

Publication number: CN112818845A
Application number: CN202110129926.0A
Authority: CN
Inventors: 杨国润; 王哲; 石建萍
Original assignee: Shenzhen Sensetime Technology Co Ltd
Current assignee: Shenzhen Sensetime Technology Co Ltd
Priority date: 2021-01-29
Filing date: 2021-01-29
Publication date: 2021-05-18

Abstract

The present disclosure provides a test method, a target object detection method, a driving control method, a device, an electronic apparatus, and a storage medium, the test method including: detecting the first sample data by using a set detection method to obtain a detection result; determining first evaluation data of the set detection method in a total distance range and second evaluation data of the set detection method in each local distance range based on the first sample data, the detection result and the labeling result of the first sample data; wherein each local range is obtained by dividing the total distance range; the total distance range is determined based on a detection range of the acquisition device corresponding to the first sample data and/or depth information of an object included in the first sample data; and determining the performance of the set detection method based on the first evaluation data and the second evaluation data.

Description

Test method, target object detection method, driving control method and device

Technical Field

The present disclosure relates to the field of deep learning technologies, and in particular, to a test method, a target object detection method, a driving control method, an apparatus, an electronic device, and a storage medium.

Background

Due to the wide application of the target detection algorithm, the target detection algorithm becomes a research hotspot which is concerned by the field of computer vision. In recent years, with the development of deep learning, research on target detection of three-dimensional 3D images has been greatly advanced, and compared with two-dimensional 2D target detection, 3D target detection combines depth information, can provide spatial scene information such as position, direction, size and the like of a target object, and has been rapidly developed in the fields of automatic driving and robots, so that 3D target detection becomes a core problem in the field of computer vision.

Therefore, it is increasingly important to provide a method for evaluating a 3D target detection algorithm more accurately.

Disclosure of Invention

In view of the above, the present disclosure provides at least a test method, a target object detection method, a driving control method, a device, an electronic apparatus, and a storage medium.

In a first aspect, the present disclosure provides a testing method, including:

detecting the first sample data by using a set detection method to obtain a detection result;

determining first evaluation data of the set detection method in a total distance range and second evaluation data of the set detection method in each local distance range based on the first sample data, the detection result and the labeling result of the first sample data; wherein each local range is obtained by dividing the total distance range; the total distance range is determined based on a detection range of the acquisition device corresponding to the first sample data and/or depth information of an object included in the first sample data;

and determining the performance of the set detection method based on the first evaluation data and the second evaluation data.

In one possible embodiment, the determining the performance of the set detection method based on the first profile and the second profile is performed by a neural network, and includes:

and determining the performance of the neural network based on the first evaluation data and the second evaluation data.

In a possible embodiment, in a case that the performance of the neural network does not reach a preset performance, the method further includes:

and training the neural network by using second sample data until the performance of the trained neural network determined based on the first evaluation data and the second evaluation data of the trained neural network reaches or exceeds the preset performance.

In a possible implementation, the first profile and the second profile include one or more of the following:

the method comprises the steps of obtaining an aerial view average accuracy BEV mAP index, a three-dimensional average accuracy 3D mAP index, a false detection rate, a missing detection rate, an average distance error, an average size error and an average orientation error.

By adopting the method, the first evaluation data and the second evaluation data comprise one or more indexes, and the performance of the detection method can be more accurately determined by setting the indexes, so that the accuracy of the detection method is improved.

In a possible embodiment, in the case that the first and/or second profile includes an average distance error, the average distance error is determined according to the following steps:

determining target sample data which meets a corresponding distance range and belongs to a real label from the first sample data based on the labeling result and the detection result of the first sample data; the target sample data of the real label is a positive sample of which the detection type indicated by the detection result is consistent with the detection type indicated by the labeling result;

determining position information of a projection detection frame of the prediction three-dimensional detection frame in the top view, which is indicated by the detection result of the target sample data, and position information of a projection real-value frame of the labeling three-dimensional detection frame in the top view, which is indicated by the labeling result of the target sample data;

determining the average distance error corresponding to the target sample data based on the position information of the projection detection frame and the position information of the projection true value frame;

and the average distance error is used for representing the position difference between the prediction three-dimensional detection frame and the labeling three-dimensional detection frame corresponding to the target sample data.

The average distance error can represent the position difference between the predicted three-dimensional detection frame and the labeled three-dimensional detection frame of the first sample data, the accuracy of the detection method for determining the position of the object can be judged through the average distance error, and the detection method can be optimized based on the average distance error, so that the optimized detection method can more accurately determine the position of the predicted three-dimensional detection frame of the object.

In a possible embodiment, in the case of an average size error being included in the first evaluation datum and/or the second evaluation datum, the average size error is determined according to the following steps:

based on the first orientation and the first central point position information of the three-dimensional detection frame marked by the marking result of the target sample data, adjusting the second orientation and the second central point position information of the three-dimensional detection frame predicted by the detection result of the target sample data, and generating an adjusted detection result corresponding to the target sample data;

calculating an intersection ratio between the labeled three-dimensional detection frame and the predicted three-dimensional detection frame indicated by the adjusted detection result based on first size information of the labeled three-dimensional detection frame indicated by the labeling result of the target sample data and second size information of the predicted three-dimensional detection frame indicated by the adjusted detection result;

determining the average size error corresponding to the target sample data based on the intersection ratio;

and the average size error is used for representing the size difference between the prediction three-dimensional detection frame and the labeling three-dimensional detection frame corresponding to the target sample data.

The average size error can represent the size difference between the predicted three-dimensional detection frame and the labeled three-dimensional detection frame corresponding to the target sample data, the accuracy of the size of the object determined by the detection method can be judged through the average size error, and the set detection method can be optimized based on the average size error, so that the optimized detection method can accurately determine the size of the predicted three-dimensional detection frame of the object.

In a possible embodiment, in the case of an average orientation error being included in the first profile and/or the second profile, the average orientation error is determined according to the following steps:

determining the average orientation error corresponding to the target sample data based on a first orientation of an annotated three-dimensional detection frame indicated by the annotation result of the target sample data and a second orientation of a predicted three-dimensional detection frame indicated by the detection result of the target sample data;

wherein the average orientation error is used for characterizing the orientation difference between the predicted three-dimensional detection box and the labeled three-dimensional detection box corresponding to the target sample data.

The average orientation error can represent the orientation difference between the predicted three-dimensional detection frame and the labeled three-dimensional detection frame corresponding to the target sample data, the accuracy of the orientation of the object determined by the detection method can be judged through the average orientation error, and the detection method can be optimized based on the average orientation error, so that the optimized detection method can more accurately determine the orientation of the predicted three-dimensional detection frame of the object.

In a possible implementation, after determining the first profile and the second profile, the method further includes:

for each detection category of at least one detection category corresponding to the set detection method, determining false detection sample data corresponding to the detection category from the first sample data and determining a false detection reason category corresponding to the false detection sample data based on a target labeling result corresponding to the detection category in a labeling result of the first sample data and a target detection result corresponding to the detection category in the detection result;

determining the performance of the set detection method based on the first evaluation data and the second evaluation data, wherein the determining comprises the following steps:

and determining the performance of the set detection method based on the false detection reason category, the first evaluation data and the second evaluation data of the false detection sample data respectively corresponding to the at least one detection category.

In the above embodiment, the false detection sample data corresponding to each detection category is determined from the first sample data, the false detection reason category of the false detection sample data corresponding to each detection category is determined, the false detection reason of the false detection sample data is clearly and clearly recognized, and the performance of the detection method is more accurately determined based on the false detection reason category, the first evaluation data and the second evaluation data respectively corresponding to at least one detection category.

for each detection category of at least one detection category corresponding to the set detection method, determining missed detection sample data corresponding to the detection category from the first sample data and determining a missed detection reason category corresponding to the missed detection sample data based on a target labeling result corresponding to the detection category in a labeling result of the first sample data and a target detection result corresponding to the detection category in the detection result;

and determining the performance of the set detection method based on the classification of the missed detection reason of the missed detection sample data, the first evaluation data and the second evaluation data which respectively correspond to the at least one detection classification.

In the above embodiment, the missed detection sample data corresponding to each detection category is determined from the first sample data, the missed detection reason category of the missed detection sample data corresponding to each detection category is determined, and the missed detection reason of the missed detection sample data is clearly and clearly recognized, so that the performance of the detection method can be more accurately determined based on the missed detection reason category, the first evaluation data and the second evaluation data respectively corresponding to at least one detection category.

for each detection category of at least one detection category corresponding to the set detection method, determining missed detection sample data corresponding to the detection category from the first sample data and determining a missed detection reason category of the missed detection sample data corresponding to the detection category based on a target labeling result corresponding to the detection category in a labeling result of the first sample data and a target detection result corresponding to the detection category in the detection result;

for each missed detection reason category corresponding to the detection category, determining target missed detection sample data belonging to the missed detection reason category from the missed detection sample data corresponding to the detection category, and generating a target missed detection thermodynamic diagram corresponding to the missed detection reason category under the detection category based on position information indicated by the labeling result of the target missed detection sample data, wherein the target missed detection thermodynamic diagram represents the missed detection probability of the missed detection reason category under the detection category at different positions within the total distance range;

and determining the performance of the set detection method based on the target missed detection thermodynamic diagram, the first evaluation data and the second evaluation data corresponding to different missed detection reason categories under each detection category.

In the foregoing embodiment, the corresponding target missing detection thermodynamic diagrams are generated for each missing detection reason category in each detection category, for example, if the detection categories are four, and the missing detection reason categories are three, 12 frames of target missing detection thermodynamic diagrams may be generated, and the missing detection probability at different positions corresponding to each missing detection reason category in each detection category may be clearly and definitely determined through the 12 frames of target missing detection thermodynamic diagrams, so that the performance of the detection method may be more accurately determined based on the multi-frame target missing detection thermodynamic diagrams, the first evaluation data, and the second evaluation data.

for each detection category of at least one detection category corresponding to the set detection method, based on a target labeling result corresponding to the detection category in the labeling results of the first sample data and a target detection result corresponding to the detection category in the detection results, determining false detection sample data corresponding to the detection category from the first sample data and determining a false detection reason category of the false detection sample data corresponding to the detection category;

for each false detection reason category corresponding to the detection category, determining target false detection sample data belonging to the false detection reason category from the false detection sample data corresponding to the detection category, and generating a target false detection thermodynamic diagram corresponding to the false detection reason category under the detection category based on position information indicated by the labeling result of the target false detection sample data, wherein the target false detection thermodynamic diagram represents false detection probabilities of the false detection reason category under the detection category at different positions within the total distance range;

and determining the performance of the set detection method based on the target false detection thermodynamic diagram, the first evaluation data and the second evaluation data corresponding to different false detection reason categories under each detection category.

In the above embodiment, a corresponding target false detection thermodynamic diagram is generated for each false detection reason category in each detection category, and the false detection probability at different positions corresponding to each false detection reason category in each detection category can be clearly and definitely determined through at least one frame of target false detection thermodynamic diagram, so that the performance of the detection method can be more accurately determined based on the multi-frame target false detection thermodynamic diagram, the first evaluation data and the second evaluation data.

The following descriptions of the effects of the apparatus, the electronic device, and the like refer to the description of the above method, and are not repeated here.

In a second aspect, the present disclosure provides a target object detection method, including:

acquiring data to be detected; the data to be detected comprises an image to be detected and/or point cloud data to be detected;

determining target detection data of each target object included in the data to be detected based on the data to be detected and a first target neural network for object detection, wherein the first target neural network is obtained by testing by using the testing method of any one of the first aspect.

In a third aspect, the present disclosure provides a running control method including:

acquiring road data acquired by a driving device in the driving process; the road data comprises road images and/or road point cloud data;

performing target detection on the road data by using a second target neural network, and determining a target object included in the road data; wherein the second target neural network is obtained by testing by using the test method of any one of the first aspect;

controlling the travel device based on the target object included in the road data.

In a fourth aspect, the present disclosure provides a test apparatus comprising:

the first detection module is used for detecting the first sample data by using a set detection method to obtain a detection result;

the first determining module is used for determining first evaluating data of the set detection method in a total distance range and second evaluating data of the set detection method in each local distance range on the basis of the first sample data, the detection result and the labeling result of the first sample data; wherein each local range is obtained by dividing the total distance range; the total distance range is determined based on a detection range of the acquisition device corresponding to the first sample data and/or depth information of an object included in the first sample data;

and the second determining module is used for determining the performance of the set detection method based on the first evaluation data and the second evaluation data.

In a fifth aspect, the present disclosure provides a target object detection apparatus, including:

the first acquisition module is used for acquiring data to be detected; the data to be detected comprises an image to be detected and/or point cloud data to be detected;

a second detection module, configured to determine, based on the data to be detected and a first target neural network used for object detection, target detection data of each target object included in the data to be detected, where the first target neural network is obtained by using the test method according to any one of the first aspect.

In a sixth aspect, the present disclosure provides a running control apparatus comprising:

the second acquisition module is used for acquiring road data acquired by the driving device in the driving process; the road data comprises road images and/or road point cloud data;

the third detection module is used for carrying out target detection on the road data by utilizing a second target neural network and determining a target object included in the road data; wherein the second target neural network is obtained by testing by using the test method of any one of the first aspect;

a control module for controlling the travel device based on the target object included in the road data.

In a seventh aspect, the present disclosure provides an electronic device, comprising: a processor, a memory and a bus, the memory storing machine-readable instructions executable by the processor, the processor and the memory communicating via the bus when the electronic device is running, the machine-readable instructions when executed by the processor performing the steps of the testing method according to the first aspect or any one of the embodiments; or a step of performing the target object detection method as described in the second aspect above; or the step of executing the running control method according to the third aspect described above.

In an eighth aspect, the present disclosure provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of the testing method according to the first aspect or any one of the embodiments described above; or a step of performing the target object detection method as described in the second aspect above; or the step of executing the running control method according to the third aspect described above.

In order to make the aforementioned objects, features and advantages of the present disclosure more comprehensible, preferred embodiments accompanied with figures are described in detail below.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings required for use in the embodiments will be briefly described below, and the drawings herein incorporated in and forming a part of the specification illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the technical solutions of the present disclosure. It is appreciated that the following drawings depict only certain embodiments of the disclosure and are therefore not to be considered limiting of its scope, for those skilled in the art will be able to derive additional related drawings therefrom without the benefit of the inventive faculty.

FIG. 1 is a flow chart of a testing method provided by an embodiment of the present disclosure;

FIG. 2 is a schematic diagram illustrating a total distance range and a local distance range in a testing method provided by an embodiment of the disclosure;

fig. 3 is a schematic flowchart illustrating a target object detection method provided by an embodiment of the present disclosure;

fig. 4 is a flow chart illustrating a driving control method according to an embodiment of the present disclosure;

FIG. 5 is a schematic diagram illustrating an architecture of a testing apparatus provided in an embodiment of the present disclosure;

fig. 6 is a schematic diagram illustrating an architecture of a target object detection apparatus provided in an embodiment of the present disclosure;

fig. 7 is a schematic diagram illustrating an architecture of a driving control device according to an embodiment of the present disclosure;

fig. 8 is a schematic structural diagram of an electronic device provided in an embodiment of the present disclosure;

fig. 9 shows a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure;

fig. 10 shows a schematic structural diagram of an electronic device provided in an embodiment of the present disclosure.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present disclosure more clear, the technical solutions of the embodiments of the present disclosure will be described clearly and completely with reference to the drawings in the embodiments of the present disclosure, and it is obvious that the described embodiments are only a part of the embodiments of the present disclosure, not all of the embodiments. The components of the embodiments of the present disclosure, generally described and illustrated in the figures herein, can be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present disclosure, presented in the figures, is not intended to limit the scope of the claimed disclosure, but is merely representative of selected embodiments of the disclosure. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the disclosure without making creative efforts, shall fall within the protection scope of the disclosure.

Due to the wide application of the target detection algorithm, the target detection algorithm becomes a research hotspot which is concerned by the field of computer vision. In recent years, with the development of deep learning, research on target detection of three-dimensional 3D images has been greatly advanced, and compared with two-dimensional 2D target detection, 3D target detection combines depth information, can provide spatial scene information such as position, direction, size and the like of a target object, and has been rapidly developed in the fields of automatic driving and robots, so that 3D target detection becomes a core problem in the field of computer vision. Therefore, in order to evaluate a 3D target detection algorithm more accurately, that is, to obtain a more accurate 3D target detection algorithm, the embodiments of the present disclosure provide a testing method and device.

The above-mentioned drawbacks are the results of the inventor after practical and careful study, and therefore, the discovery process of the above-mentioned problems and the solutions proposed by the present disclosure to the above-mentioned problems should be the contribution of the inventor in the process of the present disclosure.

The technical solutions in the present disclosure will be described clearly and completely with reference to the accompanying drawings in the present disclosure, and it is to be understood that the described embodiments are only a part of the embodiments of the present disclosure, and not all of the embodiments. The components of the present disclosure, as generally described and illustrated in the figures herein, may be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present disclosure, presented in the figures, is not intended to limit the scope of the claimed disclosure, but is merely representative of selected embodiments of the disclosure. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the disclosure without making creative efforts, shall fall within the protection scope of the disclosure.

It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures.

For the convenience of understanding the embodiments of the present disclosure, a test method, a target object detection method, and a driving control method disclosed in the embodiments of the present disclosure will be described in detail first. An execution subject of the testing method, the target object detection method, or the driving control method provided by the embodiment of the present disclosure is generally a computer device having a certain computing capability, and the computer device includes: a terminal device, which may be a User Equipment (UE), a mobile device, a User terminal, a cellular phone, a cordless phone, a Personal Digital Assistant (PDA), a handheld device, a computing device, a vehicle mounted device, a wearable device, or a server or other processing device. In some possible implementations, the test method, the target object detection method, or the driving control method may be implemented by a processor calling computer readable instructions stored in a memory.

Referring to fig. 1, a schematic flow chart of a testing method provided in the embodiment of the present disclosure is shown, the method includes S101-S103, where:

s101, detecting the first sample data by using a set detection method to obtain a detection result;

s102, determining first evaluation data of a set detection method in a total distance range and second evaluation data of each local distance range based on the first sample data, the detection result and the labeling result of the first sample data; wherein each local range is obtained by dividing the total distance range; the total distance range is determined based on the detection range of the acquisition equipment corresponding to the first sample data and/or the depth information of the object included in the first sample data;

s103, determining the performance of the set detection method based on the first evaluation data and the second evaluation data.

In the method, the first evaluation data of the set detection method in the total distance range and the second evaluation data of the set detection method in each local distance range are determined by using the first sample data, the obtained detection result and the labeling result of the first sample data, and because the performance of the detection method is related to the distance range of the object, namely, when the detection method detects images corresponding to different local distance ranges, the accuracy of the detection results corresponding to different local distance ranges is inconsistent, for example, the accuracy in the remote range is low, and the accuracy in the near range is high, the performance of the detection method can be more accurately determined by using the first evaluation data in the total distance range and the second evaluation data corresponding to each local distance range, and the accuracy of the detection method is improved.

S101 to S103 will be specifically described below.

For S101:

the first sample data may be data in an image format, point cloud data, or the like. The first sample data carries a labeling result, for example, the labeling result of the first sample data may include category information, position information, size information, orientation information, and the like of a labeled three-dimensional detection frame corresponding to each target object included in the first sample data.

The set detection method may be any method that can detect the target object in the data. For example, the set detection method may be performed by a neural network, which may detect the neural network for a three-dimensional target.

For example, the acquired first sample data carrying the labeling result may be input into a neural network that performs a detection method, the first sample data is detected, and a detection result corresponding to the first sample data is determined. The detection result may include category information, position information, size information, orientation information, confidence, and the like of the predicted three-dimensional detection frame corresponding to the first sample data.

In a possible implementation, the set detection method is executed by a neural network, and the performance of the set detection method is determined based on the first evaluation data and the second evaluation data, and the method includes: and determining the performance of the neural network based on the first evaluation data and the second evaluation data.

The first profile and the second profile can be used to determine the performance of the neural network. For example, the accuracy of the neural network can be determined by using the first evaluation data and the second evaluation data, and the performance of the neural network can be evaluated by using the accuracy; alternatively, the first evaluation data and the second evaluation data can be used to determine a loss value of the neural network, and the loss value can be used to evaluate the performance of the neural network.

In a possible embodiment, in a case that the performance of the neural network does not reach a preset performance, the method further includes: and training the neural network by using the second sample data until the performance of the trained neural network determined based on the first evaluation data and the second evaluation data of the trained neural network reaches or exceeds the preset performance.

The second sample data may be the same as or different from the first sample data. When it is determined that the performance of the neural network does not reach the preset performance, for example, when the accuracy of the neural network is smaller than the set accuracy threshold, the neural network may be trained for multiple rounds using the second sample data until the performance of the trained neural network determined based on the first evaluation data and the second evaluation data of the trained neural network reaches or exceeds the preset performance.

For S102 and S103:

here, the total distance range may be determined based on a detection range of the acquisition device corresponding to the first sample data and/or depth information of the object included in the first sample data. Wherein, the collection equipment can be a monocular camera, a binocular camera, a laser radar, a radar and the like. For example, the total distance range may be determined based on the farthest detection range of the acquisition device; when the farthest detection range of the acquisition device is 90 meters, the total distance range may be determined to be [ -90m, 90m ]. Or, the total distance range can be determined according to the optimal detection range of the acquisition equipment; for example, when the farthest detection range of the acquisition device is 90 meters, and the optimal detection range is 80 meters, that is, the range of 80 meters to 90 meters, the accuracy of the data acquired by the acquisition device is low, and at this time, the total distance range can be determined to be [ -80m, 80m ].

Alternatively, the total distance range may be determined from depth information of the object included in the first sample data. For example, if the farthest depth information of the object in the first sample data is 100 meters, the total distance range is determined to be [ -100m, 100m ].

Or, the total distance range may also be determined based on the detection range of the acquisition device corresponding to the first sample data and the depth information of the object included in the first sample data, for example, the first total distance range is determined based on the detection range of the acquisition device corresponding to the first sample data; and determining a second total distance range based on the depth information of the object included in the first sample data, and determining the intersection or union of the first total distance range and the second total distance range as the total distance range corresponding to the set detection method.

After the total distance range is determined, the total distance range may be divided into local ranges, and the dividing may be uniform or non-uniform. For example, when the total distance range is [ -90m, 90m ], each of the divided local distance ranges may include a short distance range [ -30m, 30m ], a middle distance range [ -60m, -30m ] and [30m, 60m ], a long distance range [60m, 90m ] and [ -90m, -60m ]. Alternatively, the divided local distance ranges may include short distance ranges [ -40m, 40m ], medium distance ranges [ -70m, -40m ] and [40m, 70m ], long distance ranges [70m, 90m ] and [ -90m, -70m ].

Referring to a schematic diagram of the total distance range and the local distance ranges in a test method shown in fig. 2, which is illustrated by taking an automatic driving scenario as an example, the target vehicle 21 is taken as a center, the total distance range is [ -90m, 90m ], and the plurality of local distance ranges may be a short distance range [ -30m, 30m ], a middle distance range [ -60m, -30m ], and [30m, 60m ], a long distance range [60m, 90m ], and [ -90m, -60m ]. The total distance range and the local distance range may be determined as needed, and are only exemplary.

Based on the first sample data, the detection result and the labeling result of the first sample data, determining first evaluation data of a set detection method in a total distance range, and determining second evaluation data corresponding to each local distance range, such as first evaluation data in the total distance range, second evaluation data in a long distance range, second evaluation data in a medium distance range, and second evaluation data in a short distance range.

Wherein the first evaluation data and the second evaluation data comprise one or more of the following: bird's Eye View (BEV) Average accuracy (mAP) index, three-dimensional (3Dimensions, 3D) Average accuracy (mAP) index, false detection rate, Average distance Error (ATE), Average Size Error (ASE), and Average Orientation Error (AOE).

In specific implementation, based on the detection result and the labeling result, first intermediate sample data (target sample data) corresponding to a True Positive (TP) tag, second intermediate sample data corresponding to a True Negative (TN) tag, third intermediate sample data corresponding to a False Positive (FP) tag, and fourth intermediate sample data corresponding to a False Negative (FN) tag in the first sample data may be determined.

When the first evaluation data and the second evaluation data include BEV maps, determining position information of a projection real-value frame of a labeled three-dimensional detection frame represented by a labeling result in a top view, determining position information of a projection detection frame of a predicted three-dimensional detection frame represented by a detection result in the top view, and determining a two-dimensional Intersection-over-unit (IoU) based on the position information of the projection real frame and the position information of the projection detection frame; based on the two-dimensional IoU, BEVMAP is determined.

When the first evaluation data and the second evaluation data comprise 3D mAP, calculating a 3D IoU between a labeling three-dimensional detection frame indicated by a labeling result of the first sample data and a prediction three-dimensional detection frame indicated by a detection result; based on the calculated 3DIoU, a 3D mAP is determined.

Wherein, when calculating BEV maps and 3D maps, different cross ratio thresholds may be set for different detection categories, for example, the cross ratio threshold may be set to 0.7 for vehicles and 0.5 for pedestrians.

When the first evaluation data and the second evaluation data comprise false detection rates, the number x of first intermediate sample data corresponding to the TP label in the first sample data can be determined_TPDetermining the number x of second intermediate sample data corresponding to the FP label in the first sample data_FPIf the false detection rate is x_TP/(x_TP+x_FP)。

When the first evaluation data and the second evaluation data comprise the missing rate, the number x of first intermediate sample data corresponding to the TP label in the first sample data can be determined_TPDetermining the number x of fourth intermediate sample data corresponding to the FN label in the first sample data_FNIf the false detection rate is x_TP/(x_TP+x_FN)。

In an alternative embodiment, in the case of an average distance error being included in the first and/or second evaluation data, the average distance error is determined according to the following steps:

step one, determining target sample data which meets a corresponding distance range and belongs to a real label from first sample data based on a labeling result and a detection result of the first sample data; the target sample data of the real label is a positive sample of which the detection type indicated by the detection result is consistent with the detection type indicated by the labeling result;

determining the position information of a projection detection frame of the prediction three-dimensional detection frame in the top view, which is indicated by the detection result of the target sample data, and the position information of a projection real-value frame of the labeling three-dimensional detection frame in the top view, which is indicated by the labeling result of the target sample data;

determining an average distance error corresponding to target sample data based on the position information of the projection detection frame and the position information of the projection true value frame; and the average distance error is used for representing the position difference between the predicted three-dimensional detection frame and the marked three-dimensional detection frame corresponding to the target sample data.

Here, target sample data corresponding to the TP tag may be determined from the first sample data based on the labeling result and the detection result, where the TP tag target sample data is a positive sample in which the detection category indicated by the detection result coincides with the detection category indicated by the labeling result. For the average distance error included in the first evaluation data, target sample data belonging to the TP tag may be determined from all the first sample data. For the average distance error included in the second evaluation data, target sample data belonging to the TP tag and satisfying the corresponding local distance range may be determined from all the first sample data. For example, the second evaluation data in the long-distance range may select candidate first sample data located in the long-distance range from the first sample data, and determine target sample data belonging to the TP tag from the candidate first sample data.

And then determining the position information of the projection detection frame of the prediction three-dimensional detection frame in the top view, which is indicated by the detection result of the target sample data, and determining the position information of the projection real-valued frame of the labeling three-dimensional detection frame in the top view, which is indicated by the labeling result corresponding to the target sample data, wherein the position information of the projection detection frame may be the position information of the central point of the projection detection frame, and the position information of the projection real-valued frame may be the position information of the central point of the projection real-valued frame.

And further, for each target object included in the target sample data, calculating a distance error between the position information of the projection detection frame corresponding to the target object and the position information of the projection true value frame, so that the distance error corresponding to each target object included in the target sample data can be obtained, and averaging the distance errors corresponding to each target object to obtain an average distance error corresponding to the target sample data.

The average distance error can represent the position difference between the predicted three-dimensional detection frame and the labeled three-dimensional detection frame of the target sample data, the accuracy of the set detection method for determining the position of the object can be judged through the average distance error, and the set detection method can be optimized based on the average distance error, so that the optimized detection method can accurately determine the position of the predicted three-dimensional detection frame of the object.

In a further alternative embodiment, in the case of an average size error being included in the first evaluation datum and/or in the second evaluation datum, the average size error is determined according to the following steps:

secondly, based on the first orientation and the first central point position information of the three-dimensional detection frame marked by the marking result of the target sample data, adjusting the second orientation and the second central point position information of the three-dimensional detection frame predicted by the detection result of the target sample data, and generating an adjusted detection result corresponding to the target sample data;

calculating the intersection and parallel ratio between the marked three-dimensional detection frame and the predicted three-dimensional detection frame indicated by the adjusted detection result based on the first size information of the marked three-dimensional detection frame indicated by the marking result of the target sample data and the second size information of the predicted three-dimensional detection frame indicated by the adjusted detection result;

determining an average size error corresponding to the target sample data based on the cross-over ratio; and the average size error is used for representing the size difference between the predicted three-dimensional detection frame and the marked three-dimensional detection frame corresponding to the target sample data.

In step one, for the average size error included in the first evaluation data, target sample data belonging to the TP tag may be determined from all the first sample data. For the average size error included in the second evaluation data, target sample data belonging to the TP tag, which satisfies the corresponding local distance range, may be determined from all the first sample data. For example, the second evaluation data in the long-distance range may select candidate first sample data located in the long-distance range from the first sample data, and determine target sample data belonging to the TP tag from the candidate first sample data.

In the second step, the first orientation and the first center point position information of the labeled three-dimensional detection frame indicated by the labeling result of the target sample data may be utilized to adjust the second orientation and the second center point position information of the predicted three-dimensional detection frame indicated by the detection result of the target sample data, for example, the second center point position information may be translated, the predicted three-dimensional detection frame may be rotated, and the adjusted detection result corresponding to the target sample data may be generated, so that the center point position and the orientation between the predicted three-dimensional detection frame indicated by the adjusted detection result and the labeled three-dimensional detection frame indicated by the labeling result are consistent.

In step three, for each target object included in the target sample data, based on the first size information of the labeled three-dimensional detection box corresponding to the target object indicated in the labeling result and the second size information of the predicted three-dimensional detection box corresponding to the target object indicated in the adjusted detection result, an intersection ratio IoU between the labeled three-dimensional detection box corresponding to the target object and the adjusted predicted three-dimensional detection box may be calculated, and then IoU of each target object included in the target sample data may be obtained.

In the fourth step, for each target object, the values 1 to IoU are determined based on IoU of the target object, and the average size error corresponding to the target sample data is obtained by averaging the values 1 to IoU corresponding to each target object.

The average size error can represent the size difference between the predicted three-dimensional detection frame and the labeled three-dimensional detection frame corresponding to the target sample data, the accuracy of the size of the object determined by the detection method can be judged through the average size error, and the detection method can be optimized based on the average size error, so that the optimized detection method can accurately determine the size of the predicted three-dimensional detection frame of the object.

In an alternative embodiment, in the case of an average orientation error being included in the first profile and/or the second profile, the average orientation error is determined according to the following steps:

determining an average orientation error corresponding to the target sample data based on a first orientation of the labeled three-dimensional detection frame indicated by the labeling result of the target sample data and a second orientation of the predicted three-dimensional detection frame indicated by the detection result of the target sample data; and the average orientation error is used for representing the orientation difference between the predicted three-dimensional detection frame and the labeled three-dimensional detection frame corresponding to the target sample data.

In step one, for the average orientation error included in the first evaluation data, target sample data belonging to the TP tag may be determined from all the first sample data. For the average orientation error included in the second evaluation data, target sample data belonging to the TP tag and satisfying the corresponding local distance range may be determined from all the first sample data.

In the second step, for each target object in the target sample data, based on the first orientation of the labeled three-dimensional detection frame of the target object indicated by the target labeling result corresponding to the target sample data and the second orientation of the predicted three-dimensional detection frame of the target object indicated by the target detection result corresponding to the target sample data, the orientation difference value corresponding to the target object is calculated, and the orientation difference values of the target objects in the target sample data are averaged to obtain an average orientation error corresponding to the target sample data.

In specific implementation, when the set detection method is the execution of the neural network, the first evaluation data and the second evaluation data can be used for evaluating the neural network after each training, when the trained neural network does not meet the evaluation condition, the first evaluation data and the second evaluation data are used for performing next training on the neural network to be trained, for example, the first evaluation data and the second evaluation data can be used for generating a loss value corresponding to the neural network to be trained, and the determined loss value and the second sample data are used for performing next training on the neural network to be trained until the performance of the trained neural network determined based on the first evaluation data and the second evaluation data of the trained neural network reaches or exceeds the preset performance.

For example, whether the neural network satisfies the evaluation condition may be determined according to the following manner:

in the first mode, the first evaluation data and the second evaluation data generated after training both meet a set first condition, for example, when the first evaluation data and the second evaluation data include BEVmAP, 3DmAP, false detection rate, and false drop rate, it is determined that the neural network meets the evaluation condition under the condition that BEVmAP and 3DmAP in the first evaluation data and the second evaluation data are both greater than a set accuracy threshold, false detection rates in the first evaluation data and the second evaluation data are both less than a set false detection threshold, and false drop rate in the first evaluation data and the second evaluation data is both less than a set false drop threshold.

And determining the score of the neural network after the training by using the generated first evaluation data and second evaluation data, and determining that the neural network after the training meets the evaluation condition when the score is greater than a set score threshold.

There are various ways to determine whether the neural network satisfies the evaluation condition, and this is merely an exemplary illustration.

In an alternative embodiment, after determining the first profile and the second profile, the method further comprises: and for each detection type in at least one detection type corresponding to the set detection method, determining false detection sample data corresponding to the detection type from the first sample data and determining a false detection reason type corresponding to the false detection sample data based on a target marking result corresponding to the detection type in the marking result of the first sample data and a target detection result corresponding to the detection type in the detection result.

Determining the performance of the set detection method based on the first evaluation data and the second evaluation data, wherein the performance of the set detection method comprises the following steps: and determining the performance of the set detection method based on the false detection reason category, the first evaluation data and the second evaluation data of the false detection sample data respectively corresponding to at least one detection category.

Here, the set at least one detection category may be determined as needed, such as in an automatic driving scenario, and the at least one detection category may include pedestrians, motor vehicles, non-motor vehicles, road markings, and the like.

After the first evaluation data and the second evaluation data are determined, for each of at least one detection category corresponding to the set detection method, based on a target labeling result and a target detection result corresponding to the detection category in the labeling result of the first sample data, false detection sample data corresponding to the detection category may be determined from the first sample data, where the false detection sample data corresponding to the detection category may include samples whose predicted detection category is the detection category but whose labeled detection category does not belong to the detection category.

And determining the false detection reason type corresponding to the false detection sample data, wherein the set false detection reason type comprises: false detection caused by insufficient matching IoU in category one, for example, a predicted three-dimensional detection frame is detected at the position A of the first sample data, but the matching IoU between the predicted three-dimensional detection frame and the labeled three-dimensional detection frame at the position A is insufficient, so that false detection is caused; class two, false detection caused by classification errors, for example, the target object a at the position a of the first sample data belongs to a pedestrian, but the predicted class of the target object a is a vehicle; category three, false detection due to the occurrence of null; for example, a pedestrian is present at the predicted position a, but substantially absent at the position a. The set false detection reason category may be set as needed, and is only an exemplary illustration here.

For example, when the false detection sample data includes the false detection sample data one and the false detection sample data two, it may be determined that the false detection reason category corresponding to the false detection sample data one may be the category one, and the false detection reason category corresponding to the false detection sample data two may be the category three.

Further, the performance of the set detection method may be determined based on the false detection reason category, the first evaluation data, and the second evaluation data of the false detection sample data corresponding to at least one detection category, respectively. For example, when the detection method is executed by the neural network, the weight corresponding to each detection category may be set for the false detection cause category of the false detection sample corresponding to at least one detection category, the loss value of the neural network is determined based on the set weight, the first evaluation data, and the second evaluation data, and the neural network is trained next time by using the determined loss value and the second sample data. Or, the parameters or the structure of the neural network to be trained can be adjusted according to the false detection reason category of the false detection sample corresponding to at least one detection category, and the adjusted neural network is trained by using the first evaluation data, the second evaluation data and the second sample data.

In an alternative embodiment, after determining the first profile and the second profile, the method further comprises: and for each detection type in at least one detection type corresponding to the set detection method, determining the missed detection sample data corresponding to the detection type from the first sample data and determining the missed detection reason type corresponding to the missed detection sample data based on the target labeling result corresponding to the detection type in the labeling result of the first sample data and the target detection result corresponding to the detection type in the detection result.

Determining the performance of the set detection method based on the first evaluation data and the second evaluation data, wherein the performance of the set detection method comprises the following steps: and determining the performance of the set detection method based on the missed detection reason category, the first evaluation data and the second evaluation data of the missed detection sample data respectively corresponding to at least one detection category.

After the first evaluation data and the second evaluation data are determined, for each detection category of at least one detection category corresponding to the neural network to be trained, based on a target labeling result and a target detection result corresponding to the detection category, missing detection sample data corresponding to the detection category may be determined from the first sample data, where the missing detection sample data corresponding to the detection category may include a sample in which the labeled detection category belongs to the detection category but the predicted detection category is not the detection category, and/or a sample in which the labeled detection category belongs to the detection category but a detection result corresponding to the detection category is not generated.

And determining the classification of the missed detection reason corresponding to the missed detection sample data, wherein the set classification of the missed detection reason comprises the following steps: class one, missing detection caused by insufficient matching IoU, for example, missing detection is caused by insufficient matching IoU between the predicted three-dimensional detection frame at the position a and the labeled three-dimensional detection frame when the predicted three-dimensional detection frame at the position a is detected in the first sample data; class two, missing detection caused by classification error, for example, the target object a at the position a of the first sample data belongs to a pedestrian, but the predicted class of the target object a is a vehicle; category three, pure missed inspection; for example, a pedestrian is present at the position a, but the detection result is not generated at the position a. The set missed detection reason category may be set as needed, and is only an exemplary illustration here.

For example, when the missed sample data includes the first missed sample data and the second missed sample data, it may be determined that the category of the missed reason corresponding to the first missed sample data may be the first category, and the category of the missed reason corresponding to the second missed sample data may be the third category.

Further, when the detection method is executed by the neural network, the performance of the neural network may be determined based on the missed detection cause category, the first evaluation data and the second evaluation data of the missed detection sample data respectively corresponding to at least one detection category. For example, data enhancement processing may be performed on the second sample data for the class of the cause of missed detection of the missed detection sample data corresponding to at least one detection class, for example, if it is determined that there are more pure missed detections (class three) in the long-distance range for the pedestrian (detection class), the number of the second sample data corresponding to the pedestrian in the long-distance range may be increased, so that the first evaluation data, the second evaluation data, and the second sample data after the data enhancement processing are used to train the neural network, and the performance of the neural network is determined again. Or, parameters or structures of the neural network can be adjusted according to the class of the cause of the missed detection sample data corresponding to at least one detection class, and the adjusted neural network is trained by using the first evaluation data, the second evaluation data and the second sample data.

In an alternative embodiment, after determining the first profile and the second profile, the method further comprises:

step one, aiming at each detection category in at least one detection category corresponding to a set detection method, determining missed detection sample data corresponding to the detection category and determining the missed detection reason category of the missed detection sample data corresponding to the detection category from first sample data based on a target marking result corresponding to the detection category in a marking result of the first sample data and a target detection result corresponding to the detection category in a detection result;

and secondly, determining target missed-detection samples belonging to the missed-detection reason category from the missed-detection sample data corresponding to the detection category according to each missed-detection reason category corresponding to the detection category, and generating a target missed-detection thermodynamic diagram corresponding to the missed-detection reason category under the detection category based on the position information indicated by the labeling result corresponding to the target missed-detection sample data, wherein the target missed-detection thermodynamic diagram represents the missed-detection probability of the missed-detection reason category under the detection category at different positions within the total distance range.

Determining the performance of the set detection method based on the first evaluation data and the second evaluation data, wherein the performance of the set detection method comprises the following steps: and determining the performance of the set detection method based on the target missed detection thermodynamic diagram, the first evaluation data and the second evaluation data corresponding to different missed detection reason categories under each detection category.

In the first step, for each detection category of at least one detection category corresponding to the neural network, based on a target labeling result and a target detection result corresponding to the detection category, missing detection sample data corresponding to the detection category is determined from the first sample data, and a missing detection reason category of the missing detection sample data corresponding to the detection category is determined. For example, when at least one detection category includes a pedestrian, a motor vehicle, and a non-motor vehicle, the detection category may be determined to be the missing detection sample data corresponding to the pedestrian and the missing detection reason category of the missing detection sample data corresponding to the pedestrian; the detection types can be determined as the detection missing sample data corresponding to the motor vehicle and the detection missing reason type of the detection missing sample data corresponding to the motor vehicle; the detection types can be determined as the detection missing sample data corresponding to the non-motor vehicle and the detection missing reason type of the detection missing sample data corresponding to the non-motor vehicle. The missed detection reason category may include: class one, missed detection due to insufficient match IoU; class II, missing detection caused by classification errors; category three, pure missed inspection.

In the second step, for each missed detection reason category corresponding to each detection category, target missed detection sample data belonging to the missed detection reason category may be determined from the missed detection sample data corresponding to the detection category, for example, target missed detection sample data a belonging to category one may be determined from the missed detection sample data corresponding to the pedestrian. Further, the target missed detection thermodynamic diagram corresponding to the missed detection cause category in the detection category may be generated based on the position information indicated by the labeling result corresponding to the target missed detection sample data, for example, the target missed detection thermodynamic diagram a corresponding to the first category (missed detection cause category) in the detection category of the pedestrian may be generated based on the position information indicated by the labeling result corresponding to the target missed detection sample data a. Further, it is known that, in the detection categories including pedestrians, motor vehicles, and non-motor vehicles, the cause of the missed detection category may include: class one, missed detection due to insufficient match IoU; class II, missing detection caused by classification errors; in the case of category three, pure missing detection, 9-frame target missing detection thermodynamic diagrams can be generated for various missing detection cause categories of the detection categories.

The target missing detection thermodynamic diagram represents the missing detection probability of the missing detection reason category in different positions in the total distance range under the detection category, namely the color of the pixel position in the target missing detection thermodynamic diagram represents the missing detection probability corresponding to the missing detection reason category under the detection category at the pixel position.

Further, the performance of the set detection method can be determined based on the target missed detection thermodynamic diagram, the first evaluation data and the second evaluation data corresponding to different missed detection reason categories under each detection category. For example, parameters or structures of the neural network may be adjusted based on target missed detection thermodynamic diagrams corresponding to different missed detection reason categories under each detection category, and the adjusted neural network may be retrained by using the first evaluation data, the second evaluation data, and the second sample data.

step one, aiming at each detection category in at least one detection category corresponding to a set detection method, determining false detection sample data corresponding to the detection category and determining the false detection reason category of the false detection sample data corresponding to the detection category from first sample data based on a target marking result corresponding to the detection category in a marking result of the first sample data and a target detection result corresponding to the detection category in a detection result;

and secondly, for each false detection reason type corresponding to the detection type, determining target false detection sample data belonging to the false detection reason type from the false detection sample data corresponding to the detection type, and generating a target false detection thermodynamic diagram corresponding to the false detection reason type under the detection type based on the position information indicated by the labeling result corresponding to the target false detection sample data, wherein the target false detection thermodynamic diagram represents the false detection probability at different positions corresponding to the false detection reason type under the detection type.

Determining the performance of the set detection method based on the first evaluation data and the second evaluation data, wherein the performance of the set detection method comprises the following steps: and determining the performance of the set detection method based on the target false detection thermodynamic diagram, the first evaluation data and the second evaluation data corresponding to different false detection reason categories under each detection category.

In the first step, for each detection category of at least one detection category corresponding to the neural network, based on the labeling result and the detection result corresponding to the detection category, false detection sample data corresponding to the detection category is determined from the first sample data, and a false detection reason category of the false detection sample data corresponding to the detection category is determined. For example, when at least one detection category includes a pedestrian, a motor vehicle, and a non-motor vehicle, the detection category may be determined to be a false detection sample data corresponding to the pedestrian and a false detection reason category of the false detection sample data corresponding to the pedestrian; the detection types can be determined to be false detection sample data corresponding to the motor vehicle and the false detection reason type of the false detection sample data corresponding to the motor vehicle; the detection types can be determined as false detection sample data corresponding to the non-motor vehicle and a false detection reason type of the false detection sample corresponding to the non-motor vehicle. The false detection reason category may include: false detection due to insufficient category one and matching IoU; class II, false detection caused by classification errors; and class three, false detection due to the occurrence of null.

In the second step, for each false detection reason type corresponding to each detection type, the target false detection sample data belonging to the false detection reason type may be determined from the false detection sample data corresponding to the detection type, for example, the target false detection sample data a belonging to the first type may be determined from the false detection sample data corresponding to the pedestrian. Further, the target false detection thermodynamic diagram corresponding to the false detection cause category under the detection category may be generated based on the position information indicated by the labeling result corresponding to the target false detection sample data, and for example, the target false detection thermodynamic diagram a corresponding to the detection category one (false detection cause category) under the detection category of a pedestrian may be generated based on the position information indicated by the labeling result corresponding to the target false detection sample a. Further, it is known that the detection categories include pedestrians, motor vehicles, and non-motor vehicles, and the false detection cause category may include: false detection due to insufficient category one and matching IoU; class II, false detection caused by classification errors; and the third category is that when false detection occurs due to null, a 9-frame target false detection thermodynamic diagram can be generated for each false detection reason category of each detection category.

The target false detection thermodynamic diagram represents false detection probabilities of false detection reason categories in different positions in a total distance range under the detection category, namely, a color at a pixel position in the target false detection thermodynamic diagram represents a false detection probability corresponding to the false detection reason category under the detection category at the pixel position.

Further, the performance of the set detection method can be determined based on the target false detection thermodynamic diagram, the first evaluation data and the second evaluation data corresponding to different false detection reason categories under each detection category. For example, parameters or structures of the trained neural network may be adjusted based on target false detection thermodynamic diagrams corresponding to different false detection reason categories under each detection category, and the adjusted neural network may be retrained by using the first evaluation data, the second evaluation data, and the first sample data.

In specific implementation, the usage manners of the target false detection thermodynamic diagrams corresponding to different false detection reason categories in each detection category and the target missed detection thermodynamic diagrams corresponding to different missed detection reason categories in each detection category may be determined as needed, which is exemplified here.

In the above embodiment, a corresponding target false detection thermodynamic diagram is generated for each false detection reason category in each detection category, and the false detection probability at different positions corresponding to each false detection reason category in each detection category can be clearly and definitely determined through at least one frame of target false detection thermodynamic diagram, so that the performance of the set detection method can be more accurately determined based on the multi-frame target false detection thermodynamic diagram, the first evaluation data and the second evaluation data.

It will be understood by those skilled in the art that in the method of the present invention, the order of writing the steps does not imply a strict order of execution and any limitations on the implementation, and the specific order of execution of the steps should be determined by their function and possible inherent logic.

Based on the same concept, referring to fig. 3, an embodiment of the present disclosure further provides a target object detection method, which includes S301-S302, where:

s301, acquiring data to be detected; the data to be detected comprises an image to be detected and/or point cloud data to be detected;

and S302, determining target detection data of each target object included in the data to be detected based on the data to be detected and a first target neural network for object detection, wherein the first target neural network is obtained after the test method is used for testing.

Here, the data to be detected may be any frame of image or any frame of point cloud data, and the data to be detected is input to a first target neural network for object detection, so as to determine target detection data of each target object included in the data to be detected. The first target neural network is obtained by testing by using the testing method disclosed by the embodiment of the disclosure.

The target object may be set as required, for example, the target object may be a pedestrian, a vehicle, an animal, a road sign, or the like. The target detection data of the target object may include position information, size information, and orientation information of a three-dimensional detection frame of the target object.

In the method, the first neural network is obtained by testing by using the testing method provided by the embodiment of the disclosure, so that the accuracy of the trained first neural network is higher, and further, the target detection data of the target object included in the data to be detected can be more accurately determined by using the first neural network.

Based on the same concept, referring to fig. 4, the disclosed embodiment further provides a driving control method, which includes S401-S403, wherein:

s401, acquiring road data acquired by a driving device in the driving process; the road data comprises road images and/or road point cloud data;

s402, performing target detection on the road data by using a second target neural network, and determining a target object included in the road data; the second target neural network is obtained after being tested by the testing method in the embodiment of the disclosure;

at S403, the travel device is controlled based on the target object included in the road data.

For example, the traveling device may be an autonomous vehicle, a vehicle equipped with an Advanced Driving Assistance System (ADAS), a robot, or the like. The road image may be an image acquired by the driving device in real time during driving. The target object may be any object that may appear on the road, for example, the target object may be a road sign such as a zebra crossing or a right turn, a signboard on which parking is prohibited, an animal or a pedestrian appearing on the road, or another vehicle on the road.

In particular, the acquired road data may be input into the second target neural network, and the target object included in the road data is determined, that is, the three-dimensional detection data of the target object included in the road data is determined, and the three-dimensional detection data of the target object may include size information, position information, orientation information, category information, and the like of the three-dimensional detection frame.

Further, the travel device is controlled based on the target object included in the road data. When the driving device is controlled, the driving device can be controlled to accelerate, decelerate, turn, brake and the like, or voice prompt information can be played to prompt a driver to control the driving device to accelerate, decelerate, turn, brake and the like.

In the method, the second neural network is obtained by testing by using the testing method provided by the embodiment of the disclosure, so that the accuracy of the second neural network is higher, the target object included in the road data can be more accurately determined by using the second neural network, and the driving device can be more accurately controlled.

Based on the same concept, an embodiment of the present disclosure further provides a testing apparatus, as shown in fig. 5, which is an architecture schematic diagram of the testing apparatus provided in the embodiment of the present disclosure, and includes a first detecting module 501, a first determining module 502, and a second determining module 503, specifically:

a first detecting module 501, configured to detect the first sample data by using a set detecting method, so as to obtain a detection result;

a first determining module 502, configured to determine, based on the first sample data, the detection result, and a labeling result of the first sample data, first evaluation data of the set detection method in a total distance range and second evaluation data of the set detection method in each local distance range; wherein each local range is obtained by dividing the total distance range; the total distance range is determined based on a detection range of the acquisition device corresponding to the first sample data and/or depth information of an object included in the first sample data;

a second determining module 503, configured to determine performance of the set detection method based on the first evaluation data and the second evaluation data.

In a possible implementation, the set detection method is executed by a neural network, and the second determining module 503, when determining the performance of the set detection method based on the first profile and the second profile, is configured to:

In a possible implementation, in a case that the performance of the neural network does not reach a preset performance, the apparatus further includes: a training module 504 to:

In a possible implementation, in a case that the first evaluation data and/or the second evaluation data includes an average distance error, the first determining module 502 is configured to determine the average distance error according to the following steps:

In a possible implementation, in the case that the first evaluation data and/or the second evaluation data include an average size error, the first determining module 502 is configured to determine the average size error according to the following steps:

In a possible implementation, in the case that the first profile and/or the second profile includes an average orientation error, the first determining module 502 is configured to determine the average orientation error according to the following steps:

In a possible implementation, after determining the first profile and the second profile, the apparatus further includes: a false detection determining module 505, configured to:

the second determining module 503, when determining the performance of the set detection method based on the first evaluation data and the second evaluation data, is configured to:

In a possible implementation, after determining the first profile and the second profile, the apparatus further includes: a missed detection determination module 506 to:

In a possible implementation, after determining the first profile and the second profile, the apparatus further includes: a missing detection thermodynamic diagram determination module 507 for:

In a possible implementation, after determining the first profile and the second profile, the apparatus further includes: a false detection thermodynamic diagram determination module 508 for:

Based on the same concept, an embodiment of the present disclosure further provides a target object detection apparatus, as shown in fig. 6, which is an architecture schematic diagram of the target object detection apparatus provided in the embodiment of the present disclosure, and includes a first obtaining module 601 and a second detecting module 602, specifically:

a first obtaining module 601, configured to obtain data to be detected; the data to be detected comprises an image to be detected and/or point cloud data to be detected;

a second detecting module 602, configured to determine, based on the data to be detected and a first target neural network used for object detection, target detection data of each target object included in the data to be detected, where the first target neural network is obtained by using the test method according to any one of the first aspect.

Based on the same concept, an embodiment of the present disclosure further provides a driving control device, as shown in fig. 7, which is a schematic structural diagram of the driving control device provided in the embodiment of the present disclosure, and includes a second obtaining module 701, a third detecting module 702, and a control module 703, specifically:

the second obtaining module 701 is used for obtaining road data collected by a driving device in the driving process; the road data comprises road images and/or road point cloud data;

a third detecting module 702, configured to perform target detection on the road data by using a second target neural network, and determine a target object included in the road data; wherein the second target neural network is obtained by testing by using the test method of any one of the first aspect;

a control module 703 for controlling the running device based on the target object included in the road data.

In some embodiments, the functions of the apparatus provided in the embodiments of the present disclosure or the included templates may be used to execute the method described in the above method embodiments, and specific implementation thereof may refer to the description of the above method embodiments, and for brevity, no further description is provided here.

Based on the same technical concept, the embodiment of the disclosure also provides an electronic device. Referring to fig. 8, a schematic structural diagram of an electronic device provided in the embodiment of the present disclosure includes a processor 801, a memory 802, and a bus 803. The memory 802 is used for storing execution instructions and includes a memory 8021 and an external memory 8022; the memory 8021 is also referred to as an internal memory, and is used for temporarily storing operation data in the processor 801 and data exchanged with an external memory 8022 such as a hard disk, the processor 801 exchanges data with the external memory 8022 through the memory 8021, and when the electronic device 800 operates, the processor 801 communicates with the memory 802 through the bus 803, so that the processor 801 executes the following instructions:

Based on the same technical concept, the embodiment of the disclosure also provides an electronic device. Referring to fig. 9, a schematic structural diagram of an electronic device provided in the embodiment of the present disclosure includes a processor 901, a memory 902, and a bus 903. The memory 902 is used for storing execution instructions, and includes a memory 9021 and an external memory 9022; the memory 9021 is also referred to as an internal memory, and is configured to temporarily store operation data in the processor 901 and data exchanged with an external memory 9022 such as a hard disk, the processor 901 exchanges data with the external memory 9022 through the memory 9021, and when the electronic device 900 is operated, the processor 901 communicates with the memory 902 through the bus 903, so that the processor 901 executes the following instructions:

determining detection data of each target object included in the data to be detected based on the data to be detected and a first target neural network for object detection, wherein the first target neural network is obtained by testing by using the testing method of any one of the first aspect.

Based on the same technical concept, the embodiment of the disclosure also provides an electronic device. Referring to fig. 10, a schematic structural diagram of an electronic device provided in the embodiment of the present disclosure includes a processor 1001, a memory 1002, and a bus 1003. The memory 1002 is used for storing execution instructions, and includes a memory 10021 and an external memory 10022; the memory 10021 is also referred to as a memory, and is used for temporarily storing operation data in the processor 1001 and data exchanged with the external memory 10022 such as a hard disk, the processor 1001 exchanges data with the external memory 10022 through the memory 10021, and when the electronic device 1000 operates, the processor 1001 and the memory 1002 communicate with each other through the bus 1003, so that the processor 1001 executes the following instructions:

In addition, the disclosed embodiment also provides a computer readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the computer program performs the steps of the test method described in the above method embodiment; or the steps of the target object detection method described in the above method embodiment; or to carry out the steps of the travel control method described in the above-mentioned method embodiment. The storage medium may be a volatile or non-volatile computer-readable storage medium.

The embodiments of the present disclosure also provide a computer program product, where the computer program product carries a program code, where instructions included in the program code may be used to execute steps of the test method described in the foregoing method embodiments, or execute steps of the target object detection method described in the foregoing method embodiments, or execute steps of the driving control method described in the foregoing method embodiments, which may be specifically referred to in the foregoing method embodiments and are not described herein again.

The computer program product may be implemented by hardware, software or a combination thereof. In an alternative embodiment, the computer program product is embodied in a computer storage medium, and in another alternative embodiment, the computer program product is embodied in a Software product, such as a Software Development Kit (SDK), or the like.

It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the system and the apparatus described above may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again. In the several embodiments provided in the present disclosure, it should be understood that the disclosed system, apparatus, and method may be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one logical division, and there may be other divisions when actually implemented, and for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of devices or units through some communication interfaces, and may be in an electrical, mechanical or other form.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present disclosure may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.

The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a non-volatile computer-readable storage medium executable by a processor. Based on such understanding, the technical solution of the present disclosure may be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present disclosure. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.

The above are only specific embodiments of the present disclosure, but the scope of the present disclosure is not limited thereto, and any person skilled in the art can easily conceive of changes or substitutions within the technical scope of the present disclosure, and shall be covered by the scope of the present disclosure. Therefore, the protection scope of the present disclosure shall be subject to the protection scope of the claims.

Claims

1. A method of testing, comprising:

2. The method according to claim 1, wherein the set testing method is performed by a neural network, and determining the performance of the set testing method based on the first profile and the second profile comprises:

3. The method of claim 2, wherein in the case that the performance of the neural network does not reach a preset performance, the method further comprises:

4. A method according to any of claims 1-3, wherein said first profile and said second profile comprise one or more of the following:

5. Method according to claim 4, characterized in that in case an average distance error is included in the first and/or second profile, the average distance error is determined according to the following steps:

6. Method according to claim 4 or 5, characterized in that in case an average size error is included in the first evaluation datum and/or the second evaluation datum, the average size error is determined according to the following steps:

7. Method according to any of claims 4 to 6, wherein, in case an average orientation error is included in the first profile and/or the second profile, the average orientation error is determined according to the following steps:

8. The method according to any one of claims 1 to 7, wherein after determining the first profile and the second profile, the method further comprises:

9. The method according to any one of claims 1 to 8, wherein after determining the first profile and the second profile, the method further comprises:

10. The method according to any one of claims 1 to 9, wherein after determining the first profile and the second profile, the method further comprises:

11. The method according to any one of claims 1 to 10, wherein after determining the first profile and the second profile, the method further comprises:

12. A target object detection method, comprising:

determining target detection data of each target object included in the data to be detected based on the data to be detected and a first target neural network for object detection, wherein the first target neural network is obtained by testing according to the testing method of any one of claims 2 to 11.

13. A travel control method characterized by comprising:

performing target detection on the road data by using a second target neural network, and determining a target object included in the road data; wherein the second target neural network is obtained by testing according to the test method of any one of claims 2 to 11;

14. A test apparatus, comprising:

15. A target object detection apparatus, comprising:

a second detection module, configured to determine, based on the data to be detected and a first target neural network used for object detection, target detection data of each target object included in the data to be detected, where the first target neural network is obtained by using the test method according to any one of claims 2 to 11.

16. A travel control device characterized by comprising:

the third detection module is used for carrying out target detection on the road data by utilizing a second target neural network and determining a target object included in the road data; wherein the second target neural network is obtained by testing according to the test method of any one of claims 2 to 11;

17. An electronic device, comprising: a processor, a memory and a bus, the memory storing machine-readable instructions executable by the processor, the processor and the memory communicating over the bus when the electronic device is operating, the machine-readable instructions when executed by the processor performing the steps of the test method of any of claims 1 to 11; or the step of performing the target object detection method of claim 12; or the steps of executing the running control method according to claim 13.

18. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the steps of the test method according to one of claims 1 to 11; or the step of performing the target object detection method of claim 12; or the steps of executing the running control method according to claim 13.