WO2022037403A1

WO2022037403A1 - Data processing method and apparatus

Info

Publication number: WO2022037403A1
Application number: PCT/CN2021/110287
Authority: WO
Inventors: 董旭; 刘兰个川; 毛云翔
Original assignee: 广州小鹏汽车科技有限公司
Priority date: 2020-08-20
Filing date: 2021-08-03
Publication date: 2022-02-24
Also published as: CN112017241A

Abstract

A data processing method and apparatus. The method comprises: obtaining camera data and radar data (101); performing target detection on the camera data to obtain a camera target object, and performing target detection on the radar data to obtain a radar target object (102); in a camera plan view, displaying the camera target object and the radar target object, and in a stereogram, displaying the camera target object and the radar target object (103); and according to the camera plan view and the stereogram, marking a matching relationship between the camera target object and the radar target object (104). By means of the method, the accurate marking of the matching relationship between the radar target and the camera target is achieved; and the precise marking of the matching relationship between the radar target and the camera target under a big data background can be achieved by using a marking method of mutual reference and mutual contrast of the camera plan view and an aerial view.

Description

A method and apparatus for data processing

The present invention claims the priority of the Chinese patent application filed on August 20, 2020, with the application number 202010843958.2 and the invention titled "A method and apparatus for data processing", the entire contents of which are incorporated herein by reference .

technical field

The present invention relates to the field of data processing, in particular to a method and device for data processing.

Background technique

At present, in autonomous driving and advanced assisted driving systems, the perception of the surrounding environment and target detection can be realized by configuring cameras and millimeter-wave radars. By matching the camera-based target detection results with the radar-based target detection results, it is possible to Take full advantage of the respective advantages of both sensors.

However, due to the large noise of radar target detection and the uncertainty of camera target detection in 3D, how to achieve accurate matching relationship annotation is an urgent problem to be solved.

SUMMARY OF THE INVENTION

In view of the above problems, it is proposed to provide a data processing method and apparatus to overcome the above problems or at least partially solve the above problems, including:

A method of data processing, the method comprising:

Obtain camera data and radar data;

Perform target detection on the camera data to obtain a camera target object, and perform target detection on the radar data to obtain a radar target object;

In the camera plan view, the camera target object and the radar target object are displayed, and in the stereoscopic view, the camera target object and the radar target object are displayed;

According to the camera plan view and the three-dimensional view, the matching relationship between the camera target object and the radar target object is marked.

Optionally, the displaying of the camera target object in the stereoscopic view includes:

determining three-dimensional information of the camera target object;

In the stereogram, the viewing volume corresponding to the camera target object is displayed, and the boundary of the viewing volume is set according to the three-dimensional information.

Optionally, after marking the matching relationship between the camera target object and the radar target object according to the camera plan view and the three-dimensional view, the method further includes:

A rationality check is performed on the matching relationship between the camera target object and the radar target object.

Optionally, performing a rationality check on the matching relationship between the camera target object and the radar target object includes:

For the camera target object and the radar target object having a matching relationship, determine the bottom coordinate information of the camera target object in the stereogram and the radar distance of the radar target object;

According to the base coordinate information and the radar distance, the matching relationship between the camera target object and the radar target object is checked for rationality.

Determine the number of camera target objects marked with a matching relationship with the same radar target object;

According to the number of the camera target objects, the matching relationship between the camera target objects and the radar target objects is checked for rationality.

Optionally, before the labeling of the matching relationship between the camera target object and the radar target object according to the camera plan view and the three-dimensional view, the method further includes:

judging whether the display of the camera target object in the camera plan view is accurate;

When it is determined that the display of the camera target object in the camera plan view is inaccurate, the camera target object is adjusted, and the display of the camera target object in the camera plan view and the stereoscopic view is updated.

Optionally, before performing target detection on the camera data to obtain a camera target object, and performing target detection on the radar data to obtain a radar target object, further comprising:

The camera data and the radar data are aligned.

Optionally, the perspective view includes a bird's-eye view.

A data processing device, the device includes:

Camera data and radar data acquisition module for acquiring camera data and radar data;

a target detection module, configured to perform target detection on the camera data to obtain a camera target object, and perform target detection on the radar data to obtain a radar target object;

a camera target object and radar target object display module, configured to display the camera target object and the radar target object in a camera plan view, and display the camera target object and the radar target object in a stereoscopic view;

The matching relationship labeling module is configured to label the matching relationship between the camera target object and the radar target object according to the camera plan view and the stereogram.

A server includes a processor, a memory, and a computer program stored on the memory and capable of running on the processor, the computer program implementing the data processing method as described above when executed by the processor.

A computer-readable storage medium stores a computer program on the computer-readable storage medium, and when the computer program is executed by a processor, implements the above-mentioned data processing method.

The embodiments of the present invention have the following advantages:

In the embodiment of the present invention, by acquiring camera data and radar data, performing target detection on the camera data to obtain the camera target object, and performing target detection on the radar data to obtain the radar target object, and then displaying the camera target object in the camera plan view And radar target object, and display the camera target object and radar target object in the stereo map, and then mark the matching relationship between the camera target object and the radar target object according to the camera plan and stereo map, and accurately mark the matching relationship between the radar target and the camera target. The relationship between the radar target and the camera target in the context of big data can be accurately matched and labeled by using the labeling method of mutual reference and comparison between the camera plan and the bird's-eye view.

The above description is only an overview of the technical solutions of the present invention, in order to be able to understand the technical means of the present invention more clearly, it can be implemented according to the content of the description, and in order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand , the following specific embodiments of the present invention are given.

Description of drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the accompanying drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description These are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can also be obtained according to these drawings without creative efforts.

1 is a flowchart of steps of a method for data processing provided by an embodiment of the present invention;

2a is a schematic diagram of an example of displaying a camera target and a radar target according to an embodiment of the present invention;

2b is a schematic diagram of another example of displaying a camera target and a radar target according to an embodiment of the present invention;

3 is a flowchart of steps of another data processing method provided by an embodiment of the present invention;

4 is a flowchart of steps of another data processing method provided by an embodiment of the present invention;

5 is a schematic diagram of an example of a matching relationship labeling process provided by an embodiment of the present invention;

FIG. 6 is a schematic structural diagram of a data processing apparatus according to an embodiment of the present invention.

specific embodiment

In order to make the purposes, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments These are some embodiments of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

Referring to FIG. 1, a flowchart of steps of a data processing method provided by an embodiment of the present invention is shown, which may specifically include the following steps:

Step 101, acquiring camera data and radar data;

In the process of labeling the radar target and the camera target, the camera data and the radar data can be acquired to perform target detection for the camera data and the radar data respectively. Further, target detection can be performed based on camera signals and radar signals, respectively.

In autonomous driving and advanced assisted driving systems, the perception of the surrounding environment and target detection can be achieved by configuring cameras and millimeter-wave radars. The camera-based target detection results are highly accurate and rich in information, but are also subject to Limited to two-dimensional detection results, the value in practical applications is limited; radar-based target detection results can obtain the three-dimensional position and velocity information of the detected object, but at the same time, it is also limited by the large noise and low accuracy of radar target detection. high impact. By matching the radar detection target and the camera detection target, the respective advantages of the two sensors can be fully utilized, and accurate three-dimensional object information can be obtained.

Since the traditional camera radar target matching scheme is a fixed rule algorithm based on human experience summary, the algorithm is mainly based on human defined logic, and is formed by logic programming by combining the multi-scenario fusion rules summed up by human experience. However, this algorithm has many limitations and shortcomings in practical applications, such as low accuracy and poor robustness.

In an example, in order to solve the above problems and improve the performance of camera radar target matching, a data-driven machine learning-based target matching algorithm can be used to achieve, while the data-driven solution needs to be based on a large amount of radar data and camera data, Moreover, the radar data and the camera data can have the matching relationship between the accurately marked camera target and the radar target, and the accurate marking tool of the present invention can realize the accurate marking of the matching relationship between the radar target and the camera target under the background of big data, and can be applied to Implementation of camera-radar target fusion algorithm based on machine learning.

Step 102, performing target detection on the camera data to obtain a camera target object, and performing target detection on the radar data to obtain a radar target object;

After acquiring the camera data and the radar data, target detection can be performed on the camera data and the radar data respectively, and then the camera target object and the radar target object can be obtained.

For example, the detection of the camera target can be completed through pre-trained model inference, or the detection can be performed by manual labeling. The present invention can use the pre-trained model for automatic detection to improve labeling efficiency.

Step 103, displaying the camera target object and the radar target object in a camera plan view, and displaying the camera target object and the radar target object in a three-dimensional view;

As an example, the stereo map may include a bird's-eye view, which may be used for cross-reference and cross-reference annotation with the camera plan.

After the camera target object and the radar target object are obtained, the camera target object and the radar target object can be displayed in the camera plane view, and the camera target object and the radar target object can be displayed in the stereogram, so as to further adopt the camera plane view and stereogram , matching and labeling the camera target object and the radar target object.

For example, the radar target (that is, the radar target object) and the camera target (that is, the camera target object) can be displayed through the camera plane and the bird's-eye view, respectively. (view frustum) represents the position of the camera target.

In an example, the representation method for radar targets displayed in the camera plane or bird's-eye view can be adjusted, such as using different colors and shapes (square or prism, etc.) to represent radar targets; different upper and lower boundaries of the visual image can be used. to represent the camera target in a bird's-eye view.

In yet another example, the specific representation of the viewing volume in the bird's-eye view can also be a triangle, and the viewing volume (a triangle with no maximum and minimum distance restrictions) or a limited viewing volume (a trapezoid with a maximum and minimum distance limit) can be used in the bird's-eye view. Position the object. Due to the uncertainty of the three-dimensional position of the target detected by the camera (such as in the depth direction), it is difficult to accurately represent the camera target on the bird's-eye view, which will affect the accurate annotation of the camera-radar matching relationship. Obtaining the limited view volume, and then using the limited view volume representation method can help reduce the three-dimensional uncertainty of the camera target, and provide the most possible labeling clues, so that accurate labeling can be achieved.

In an embodiment of the present invention, the displaying of the camera target object in the stereoscopic view may include the following sub-steps:

Sub-step 11, determine the three-dimensional information of the camera target object;

After the camera target object is obtained, corresponding three-dimensional information can be obtained for the camera target object.

For example, the three-dimensional information of the detected object in the camera plane can be estimated, and the specific methods for obtaining the three-dimensional information of the camera target object can be various, for example, it can be estimated based on the length and width of the detected bounding box, or it can be based on inverse perspective transformation. (inverse perspective transform, IPM), using the bottom edge of the detection bounding box to calculate the actual three-dimensional position of the detected target, the inverse perspective transformation method can be calculated in the following ways:

Among them, the sum in this formula can be the external parameter matrix and the internal parameter matrix of the camera sensor, respectively, and Z can be set to 0, so that the detected target of the camera plane can be projected on the ground, and then the depth difference of the camera detected target can be eliminated. Uncertainty.

Sub-step 12, in the stereoscopic view, display the visual volume corresponding to the camera target object, and set the boundary of the visual volume according to the three-dimensional information.

After the three-dimensional information of the camera target object is determined, the camera target object can be displayed in a stereogram, which can display the view volume corresponding to the camera target object, and can set the boundary for the view volume according to the three-dimensional information of the camera target object.

For example, the viewing volume can be obtained by adding upper and lower boundaries to obtain a limited viewing volume, and the upper and lower boundaries can represent the actual size range of the detected object, such as 1 meter and 5 meters. as shown in Figure 2a).

In an example, the orientation of the camera plan view and the bird's-eye view can be adjusted, as shown in Figure 2a, the display method can be that the camera plan view is displayed on the left and the bird's eye view is displayed on the right; as shown in Figure 2b, the camera plan view can also be displayed. On the top, the bird's-eye view is displayed on the bottom, which is not limited in the present invention. In different display modes, the two perspectives of the camera plan view and the bird's-eye view can be switched, for example, a shortcut key can be used to switch.

Step 104: Mark the matching relationship between the camera target object and the radar target object according to the camera plan view and the three-dimensional view.

In the specific implementation, the camera target object and the radar target object are displayed in the camera plan view and stereo map, and then the matching relationship between the camera target object and the radar target object can be marked according to the camera plan view and stereo map.

For the camera signal, since the camera signal is the information of the two-dimensional plane projected by the three-dimensional object through perspective transformation, the camera signal has three-dimensional uncertainty (mainly in the depth direction), which is the camera signal. Due to the imaging principle, it is difficult to overcome, resulting in three-dimensional uncertainty in the detection target of the camera.

For radar signals, due to the limitation of radar technology on the market, the radar detection target often contains noise, and is also limited by the radar hardware configuration, the radar detection target is highly uncertain, and the noise and uncertainty lead to the camera. The ambiguity in the matching relationship labeling of radar targets affects the accurate labeling. The ambiguity can be manifested in the following aspects:

(1) On the camera plane, radar targets generated by objects of different depths may be projected to the same position;

(2) The projection of the radar target in the height direction of the camera plane is inaccurate;

(3) The projection of the camera target in the depth direction of the bird's-eye view is inaccurate.

By using the camera plan and bird's-eye view at the same time for the matching and labeling method of the camera and radar perception results, the camera plan and bird's-eye view can be referred to and compared with each other, so that the geometric relationship between the three-dimensional objects can be better combined to obtain the results between the camera plane and the bird's eye view. Different projections in the bird's-eye view can present richer semantic relations, geometric relations, etc. as the basis for labeling reference, so as to achieve accurate matching labeling.

In the embodiment of the present invention, by acquiring camera data and radar data, performing target detection on the camera data to obtain the camera target object, and performing target detection on the radar data to obtain the radar target object, and then displaying the camera target object in the camera plan view and radar target object, and display the camera target object and radar target object in the stereo map, and then mark the matching relationship between the camera target object and the radar target object according to the camera plan and stereo map, and realize the accurate marking of the matching relationship between the radar target and the camera target. The relationship between the radar target and the camera target in the context of big data can be accurately matched and labeled by using the method of mutual reference and comparison between the camera plan and the bird's-eye view.

Referring to FIG. 3, a flowchart of steps of a data processing method provided by an embodiment of the present invention is shown, which may specifically include the following steps:

Step 301, acquiring camera data and radar data;

Step 302, performing target detection on the camera data to obtain a camera target object, and performing target detection on the radar data to obtain a radar target object;

Step 303, displaying the camera target object and the radar target object in a camera plan view, and displaying the camera target object and the radar target object in a stereoscopic view;

After obtaining the camera target object and the radar target object, the camera target object and the radar target object can be displayed in the camera plane view, and the camera target object and the radar target object can be displayed in the stereogram, so as to further adopt the camera plane view and stereogram , matching and labeling the camera target object and the radar target object.

Step 304, marking the matching relationship between the camera target object and the radar target object according to the camera plan view and the three-dimensional view;

Step 305: Check the rationality of the matching relationship between the camera target object and the radar target object.

In the specific implementation, a sanity check can be performed on the matching relationship between the camera target object and the radar target object. For example, after the labeling of all camera targets is completed, a labeling rationality check can be performed. The content can include two parts: one is to judge whether there is a self-contradiction in the marking result; the other is to judge whether the marking is complete.

In an example, there can be various methods for judging whether there is a self-contradiction, and any two matching camera targets can be checked to determine whether the order of their bottom coordinates and the radar distance are consistent; it can be detected whether there is the same The radar targets are connected multiple times, and they are all marked with a definite relationship for judgment.

In yet another example, to determine whether the annotation is complete, it may be detected whether all the camera objects are marked for determination.

In an embodiment of the present invention, step 305 may include the following sub-steps:

For the camera target object and the radar target object with a matching relationship, determine the bottom coordinate information of the camera target object in the stereogram and the radar distance of the radar target object; according to the bottom coordinate information and the radar distance, and the matching relationship between the camera target object and the radar target object is checked for rationality.

In the process of rationality check, for the camera target object and radar target object with matching relationship, the bottom coordinate information of the camera target object in the stereo map and the radar distance of the radar target object can be determined, and then the bottom coordinate information and the radar distance of the radar target object can be determined. The radar distance is used to judge whether there is a self-contradiction in the marking results, so as to check the rationality of the matching relationship between the camera target object and the radar target object.

For example, any two matching camera targets can be checked to determine whether the order of their bottom coordinates is consistent with the radar distance. Due to the principle of inverse perspective transformation, the larger the bottom coordinate value is, the closer the corresponding target radar distance is. , the algorithm can issue a false warning if any two targets do not meet this criterion.

Determine the number of camera target objects marked with a matching relationship with the same radar target object; according to the number of camera target objects, perform a rationality check on the matching relationship between the camera target object and the radar target object.

In the process of rationality check, the number of camera target objects marked with a matching relationship with the same radar target object can be determined, and then according to the number of camera target objects, the completeness of the labeling can be judged, so as to determine whether the camera target object and the radar target object are complete. The matching relationship is checked for plausibility.

Due to the three-dimensional uncertainty of the camera target and the noise problem of the radar target, it is difficult to accurately label, and the rationality check can be a key step to achieve accurate labeling. Geometric constraints such as ordinal can be fully utilized in order to find errors in annotation.

In the embodiment of the present invention, by acquiring camera data and radar data, performing target detection on the camera data to obtain the camera target object, and performing target detection on the radar data to obtain the radar target object, and then displaying the camera target object in the camera plan view and radar target object, and display the camera target object and radar target object in the stereo map, and then mark the matching relationship between the camera target object and the radar target object according to the camera plan and stereo map, and determine the matching relationship between the camera target object and the radar target object. The rationality check is carried out to accurately mark the matching relationship between the radar target and the camera target. By using the labeling method of mutual reference and comparison between the camera plan and bird's-eye view, and carrying out the labeling rationality check, the radar target can be targeted against the background of big data. Annotate the exact matching relationship with the camera target.

Referring to FIG. 4, a flowchart of steps of another data processing method provided by an embodiment of the present invention is shown, which may specifically include the following steps:

Step 401, acquiring camera data and radar data;

Step 402, aligning the camera data and the radar data;

After the camera data and radar data are obtained, time alignment and space alignment processing can be performed on the camera data and the radar data, so as to further use the aligned and calibrated data for subsequent processing. For example, radar detection results can be spatially and temporally aligned to the camera plane.

Step 403, performing target detection on the camera data to obtain a camera target object, and performing target detection on the radar data to obtain a radar target object;

Step 404, displaying the camera target object and the radar target object in a camera plan view, and displaying the camera target object and the radar target object in a three-dimensional view;

Step 405, judging whether the display of the camera target object in the camera plan view is accurate;

After displaying the camera target object and the radar target object in the camera plan view, and displaying the camera target object and the radar target object in the stereo view, it can be judged whether the display of the camera target object in the camera plan view is accurate.

Step 406, when it is determined that the camera target object is displayed inaccurately in the camera plan view, adjust the camera target object, and update the display of the camera target object in the camera plan view and the stereoscopic view;

In the process of judging whether the display of the camera target object in the camera plan view is accurate, the camera target object can be adjusted when it is determined that the display of the camera target object in the camera plan view is inaccurate, and the camera target object can be updated in the camera plan view and stereo map. display in . By labeling each camera target one by one, the matching radar points corresponding to the camera target can be selected.

For example, a camera target can be checked for accuracy, and in the event of inaccuracy, the target can be chosen to be ignored or manually adjusted. When selecting the adjustment target, the view volume corresponding to the camera target object in the bird's eye view can also be adjusted accordingly.

In an example, related configurations can be made for the annotation tool, for example, whether to allow the annotator to adjust the bounding box of the camera target object in the camera plan view.

In yet another example, when the camera target is inaccurate and no adjustment is selected, or the target is heavily occluded, or the target location is far away, the target can be ignored. For each radar point that can be matched, it can be selected whether the matching relationship of the camera radar target based on the radar point is determined.

Step 407: Mark the matching relationship between the camera target object and the radar target object according to the camera plan view and the three-dimensional view.

In order to enable those skilled in the art to better understand the above steps, the embodiment of the present invention is exemplified below with reference to FIG. 5 , but it should be understood that the embodiment of the present invention is not limited thereto.

1. Millimeter wave radar data collection;

2. Target detection based on millimeter wave radar;

3. Spatial and temporal alignment of radar detection results to the camera plane;

4. Camera data collection;

5. Camera-based target detection;

6. Three-dimensional information estimation of the detected target of the camera;

7. The display of radar detection target on the camera plane: using radar point representation;

8. The display of the camera detection target on the camera plane: using the bounding box representation;

9. The display of the radar detection target under the bird's-eye view: using the radar point and radar speed arrow representation;

10. The display of the camera detection target under the bird's-eye view: using object point and visual volume representation;

11. Check and label each camera detection target one by one;

12. Is the camera target inaccurate? When it is inaccurate, adjust the camera target, and update the display of the adjusted camera target; when it is accurate, can further detect the camera target be ignored? When it can be ignored, ignore the target, and when it cannot be ignored, proceed to the next step of detection;

13. Does the camera target have no radar point?

14. When there are no radar points, check if all camera targets are labeled?

15. If not, you can hide the radar point that the camera detects that the target has been matched and determined;

16. Investigate the next camera detection target;

17. When the camera target has radar points, mark the radar points that match the camera target one by one, and select whether to confirm each matching point;

18. Does the labeling result meet the rationality check? When not satisfied, return to inspect and label each camera detection target one by one; when the rationality check is satisfied, end.

In the embodiment of the present invention, by acquiring camera data and radar data, aligning the camera data and radar data, and then performing target detection on the camera data to obtain the camera target object, and performing target detection on the radar data to obtain the radar target object, In the camera plan view, the camera target object and the radar target object are displayed, and in the stereoscopic view, the camera target object and the radar target object are displayed, and then it is judged whether the display of the camera target object in the camera plan view is accurate. When the display is inaccurate, adjust the camera target object, and update the display of the camera target object in the camera plan view and stereo map. According to the camera plan view and stereo map, mark the matching relationship between the camera target object and the radar target object, and compare the camera target object. The matching relationship with the radar target object is checked for rationality, and the matching relationship between the radar target and the camera target can be accurately marked. By using the labeling method of mutual reference and comparison between the camera plan and the bird's-eye view, it can target the radar target under the background of big data. Annotate the exact matching relationship with the camera target.

It should be noted that, for the sake of simple description, the method embodiments are described as a series of action combinations, but those skilled in the art should know that the embodiments of the present invention are not limited by the described action sequences, because According to embodiments of the present invention, certain steps may be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

Referring to FIG. 6 , a schematic structural diagram of a data processing apparatus provided by an embodiment of the present invention is shown, which may specifically include the following modules:

a camera data and radar data acquisition module 601 for acquiring camera data and radar data;

A target detection module 602, configured to perform target detection on the camera data to obtain a camera target object, and perform target detection on the radar data to obtain a radar target object;

a camera target object and radar target object display module 603, configured to display the camera target object and the radar target object in a camera plan view, and display the camera target object and the radar target object in a stereoscopic view;

The matching relationship labeling module 604 is configured to label the matching relationship between the camera target object and the radar target object according to the camera plan view and the three-dimensional view.

In an embodiment of the present invention, the camera target object and radar target object display module 603 includes:

a three-dimensional information determination submodule, used for determining the three-dimensional information of the camera target object;

The visual volume boundary setting sub-module is configured to display the visual volume corresponding to the camera target object in a stereoscopic image, and set the boundary of the visual volume according to the three-dimensional information.

In an embodiment of the present invention, the device further includes:

The rationality checking module is used for checking the rationality of the matching relationship between the camera target object and the radar target object.

In an embodiment of the present invention, the rationality checking module includes:

The bottom coordinate information and radar distance determination sub-module is used to determine the bottom coordinate information of the camera target object in the stereo image and the radar target object of the radar target object for the camera target object and the radar target object with a matching relationship distance;

The first rationality checking sub-module is configured to perform a rationality check on the matching relationship between the camera target object and the radar target object according to the base coordinate information and the radar distance.

The sub-module for determining the number of camera target objects is used to determine the number of camera target objects marked with a matching relationship with the same radar target object;

The second rationality checking sub-module is configured to perform a rationality check on the matching relationship between the camera target object and the radar target object according to the number of the camera target objects.

In an embodiment of the present invention, the device further includes:

a judgment module for judging whether the display of the camera target object in the camera plan view is accurate;

A camera target object adjustment module, configured to adjust the camera target object when it is determined that the camera target object is displayed inaccurately in the camera plan view, and update the camera target object in the camera plan view and the camera target object Display in the stereogram.

In an embodiment of the present invention, the device further includes:

an alignment module, configured to align the camera data and the radar data.

In an embodiment of the present invention, the three-dimensional view includes a bird's-eye view.

In the embodiment of the present invention, by acquiring camera data and radar data, performing target detection on the camera data to obtain the camera target object, and performing target detection on the radar data to obtain the radar target object, and then displaying the camera target object in the camera plan view and radar target object, and display the camera target object and radar target object in the stereo map, and then mark the matching relationship between the camera target object and the radar target object according to the camera plan and stereo map, and realize the accurate marking of the matching relationship between the radar target and the camera target. The relationship between the radar target and the camera target in the context of big data can be accurately matched and labeled by using the labeling method of mutual reference and comparison between the camera plan and the bird's-eye view.

An embodiment of the present invention also provides a server, which may include a processor, a memory, and a computer program stored in the memory and capable of running on the processor. When the computer program is executed by the processor, the above data processing method is implemented.

An embodiment of the present invention further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the above data processing method is implemented.

As for the apparatus embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and reference may be made to the partial description of the method embodiment for related parts.

The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same and similar parts between the various embodiments may be referred to each other.

It should be understood by those skilled in the art that the embodiments of the present invention may be provided as a method, an apparatus, or a computer program product. Accordingly, embodiments of the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media having computer-usable program code embodied therein, including but not limited to disk storage, CD-ROM, optical storage, and the like.

Embodiments of the present invention are described with reference to flowcharts and/or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the present invention. It will be understood that each process and/or block in the flowchart illustrations and/or block diagrams, and combinations of processes and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to the processor of a general purpose computer, special purpose computer, embedded processor or other programmable data processing terminal equipment to produce a machine that causes the instructions to be executed by the processor of the computer or other programmable data processing terminal equipment Means are created for implementing the functions specified in a flow or flows of the flowcharts and/or a block or blocks of the block diagrams.

These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing terminal equipment to operate in a particular manner, such that the instructions stored in the computer readable memory result in an article of manufacture comprising instruction means, the The instruction means implement the functions specified in the flow or flow of the flowcharts and/or the block or blocks of the block diagrams.

These computer program instructions can also be loaded on a computer or other programmable data processing terminal equipment, so that a series of operational steps are performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby executing on the computer or other programmable terminal equipment The instructions executed on the above provide steps for implementing the functions specified in the flowchart or blocks and/or the block or blocks of the block diagrams.

Although preferred embodiments of the embodiments of the present invention have been described, additional changes and modifications to these embodiments may be made by those skilled in the art once the basic inventive concepts are known. Therefore, the appended claims are intended to be construed to include the preferred embodiments as well as all changes and modifications that fall within the scope of the embodiments of the present invention.

Finally, it should also be noted that in this document, relational terms such as first and second are used only to distinguish one entity or operation from another, and do not necessarily require or imply these entities or that there is any such actual relationship or sequence between operations. Moreover, the terms "comprising", "comprising" or any other variation thereof are intended to encompass non-exclusive inclusion, such that a process, method, article or terminal device comprising a list of elements includes not only those elements, but also a non-exclusive list of elements. other elements, or also include elements inherent to such a process, method, article or terminal equipment. Without further limitation, an element defined by the phrase "comprises a..." does not preclude the presence of additional identical elements in the process, method, article, or terminal device that includes the element.

The provided method and device for data processing have been introduced in detail above. Specific examples are used in this paper to illustrate the principles and implementations of the present invention. The descriptions of the above embodiments are only used to help understand the present invention. method and its core idea; at the same time, for those skilled in the art, according to the idea of the present invention, there will be changes in the specific implementation and application scope. Invention limitations.

Claims

A method for data processing, characterized in that the method comprises:

Obtain camera data and radar data;

Perform target detection on the camera data to obtain a camera target object, and perform target detection on the radar data to obtain a radar target object;

In the camera plan view, the camera target object and the radar target object are displayed, and in the stereoscopic view, the camera target object and the radar target object are displayed;

According to the camera plan view and the three-dimensional view, the matching relationship between the camera target object and the radar target object is marked.
The method according to claim 1, wherein the displaying the camera target object in a stereoscopic image comprises:

determining three-dimensional information of the camera target object;

In the stereogram, the viewing volume corresponding to the camera target object is displayed, and the boundary of the viewing volume is set according to the three-dimensional information.
The method according to claim 1 or 2, wherein after the marking the matching relationship between the camera target object and the radar target object according to the camera plan view and the three-dimensional view, the method further comprises:

A rationality check is performed on the matching relationship between the camera target object and the radar target object.
The method according to claim 3, wherein the performing a rationality check on the matching relationship between the camera target object and the radar target object comprises:

For the camera target object and the radar target object having a matching relationship, determine the bottom coordinate information of the camera target object in the stereogram and the radar distance of the radar target object;

According to the base coordinate information and the radar distance, the matching relationship between the camera target object and the radar target object is checked for rationality.
The method according to claim 3, wherein the performing a rationality check on the matching relationship between the camera target object and the radar target object comprises:

Determine the number of camera target objects marked with a matching relationship with the same radar target object;

According to the number of the camera target objects, the matching relationship between the camera target objects and the radar target objects is checked for rationality.
The method according to claim 1, wherein before the marking the matching relationship between the camera target object and the radar target object according to the camera plan view and the three-dimensional view, the method further comprises:

judging whether the display of the camera target object in the camera plan view is accurate;

When it is determined that the display of the camera target object in the camera plan view is inaccurate, the camera target object is adjusted, and the display of the camera target object in the camera plan view and the stereoscopic view is updated.
The method according to claim 1, wherein, before performing target detection on the camera data to obtain a camera target object, and performing target detection on the radar data to obtain a radar target object, the method further comprises:

The camera data and the radar data are aligned.
The method of claim 1, wherein the perspective view comprises a bird's-eye view.
A data processing device, characterized in that the device comprises:

Camera data and radar data acquisition module for acquiring camera data and radar data;

a target detection module, configured to perform target detection on the camera data to obtain a camera target object, and perform target detection on the radar data to obtain a radar target object;

a camera target object and radar target object display module, configured to display the camera target object and the radar target object in a camera plan view, and display the camera target object and the radar target object in a stereoscopic view;

The matching relationship labeling module is configured to label the matching relationship between the camera target object and the radar target object according to the camera plan view and the stereogram.
A server, characterized by comprising a processor, a memory, and a computer program stored on the memory and capable of running on the processor, the computer program being executed by the processor to implement claims 1 to 1 The data processing method of any one of 8.
A computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the data processing according to any one of claims 1 to 8 is implemented. method.