WO2023040137A1

WO2023040137A1 - Data processing

Info

Publication number: WO2023040137A1
Application number: PCT/CN2022/070559
Authority: WO
Inventors: 黄超; 张�浩
Original assignee: 上海仙途智能科技有限公司
Priority date: 2021-09-16
Filing date: 2022-01-06
Publication date: 2023-03-23
Also published as: CN115248441A

Abstract

Provided are a data processing method, a radar calibration apparatus, a terminal and a computer-readable storage medium. The method comprises: acquiring a point cloud map of a first target scene, and point cloud data obtained by means of a plurality of radars scanning the first target scene (101); on the basis of the acquired point cloud data corresponding to each radar, and the point cloud map of the first target scene, determining a first transfer matrix corresponding to each radar (102); taking any one of the plurality of radars as a reference radar, and determining, on the basis of the first transfer matrix corresponding to the reference radar, a second transfer matrix corresponding to radars other than the reference radar among the plurality of radars, such that data collected by each radar can be mapped to a coordinate system of the reference radar (103); and on the basis of the reference radar and the second transfer matrix corresponding to the other radars, further performing data fusion on point cloud data obtained by means of the plurality of radars scanning a second target scene (104), such that the fusion of data acquired by a plurality of radars can be implemented.

Description

data processing

technical field

This description relates to the technical field of automatic driving, and in particular to methods, devices, terminals and media for data processing.

Background technique

As a product of the integration of the automotive industry and next-generation information technologies such as artificial intelligence and big data, autonomous driving technology can reduce traffic accidents and improve driving safety.

In the autonomous driving technology, the high-wire beam lidar is generally used as the main sensor of the perception system of the autonomous vehicle, so that the laser beam is emitted in multiple directions through the high-wire beam lidar, and then the location of the autonomous vehicle is determined according to the time when the laser beam is received. The distribution of obstacles in the environment.

However, due to the high cost of high-wire beam lidar, the cost of self-driving vehicles is high. In view of the above situation, the solution of using multiple low-wire beam lidars instead of one high-wire beam lidar came into being. However, each laser The origin and positive direction of the coordinate system of the radar are not the same, so that the data acquired by multiple lidars cannot be fused.

Contents of the invention

In order to overcome the problems existing in the related technologies, this specification provides the following data methods, devices, terminals and media.

According to the first aspect of the embodiments of this specification, there is provided a data processing method, the method comprising: acquiring a point cloud map of the first target scene and point cloud data obtained by scanning the first target scene with multiple radars; The point cloud data corresponding to each radar and the point cloud map of the first target scene are obtained, and the first transfer matrix corresponding to each radar is determined. The first transfer matrix is the transfer of the coordinate system of the corresponding radar relative to the coordinate system of the first target scene matrix; using any one of the multiple radars as a reference radar, based on the first transfer matrix corresponding to the reference radar, determine the second transfer matrix corresponding to other radars in the multiple radars except the reference radar, and the second transfer matrix is the corresponding radar A transfer matrix of the coordinate system relative to the coordinate system of the reference radar; based on the second transfer matrix corresponding to the reference radar and other radars, data fusion is performed on the point cloud data obtained by scanning the second target scene by multiple radars.

In some embodiments, obtaining the point cloud map of the first target scene includes: based on the point cloud data collected by the preset radar located in the first target scene, determining the equation of motion for the target object equipped with the preset radar and an observation equation, the motion equation is used to indicate the relationship between the position of the target object and the motion data of the target object, and the observation equation is used to indicate the relationship between the position of the preset point in the first target scene and the position of the target object; Based on the motion equation of the target object and the observation equation, the position of the target object and the positions of multiple preset points in the first target scene are determined to obtain a point cloud map of the first target scene.

In some embodiments, based on the obtained point cloud data corresponding to each radar and the point cloud map of the first target scene, determining the first transfer matrix corresponding to each radar includes: for any radar in the plurality of radars, obtaining The target parameters of any radar meet the target parameter value of the setting condition, and the setting condition is the matching degree of the position of the point corresponding to the point cloud data of any radar with the position of the point in the point cloud map of the first target scene maximum; based on the target parameter value and the point cloud map of the first target scene, determine the first transfer matrix corresponding to any radar.

In some embodiments, for any radar among the plurality of radars, obtaining the target parameter value whose target parameter satisfies the set condition of any radar includes: responding to the parameter value adjustment operation on the target parameter, based on the adjusted parameter value , to display the point cloud map corresponding to the point cloud data of any radar; in response to the submission operation of the parameter value of the target parameter, obtain the current parameter value of the target parameter as the target parameter value of the target parameter.

In some embodiments, the first transfer matrix includes a first rotation matrix and a first translation matrix; based on the obtained point cloud data corresponding to each radar and the point cloud map of the first target scene, determine the first rotation matrix corresponding to each radar. The transfer matrix includes: for any radar among multiple radars, the point cloud data corresponding to any radar and the point cloud data corresponding to the target parameters are used as the intermediate point cloud data corresponding to any radar, based on the intermediate point cloud data, the first A point cloud map of a target scene and a target error function, determine a first rotation matrix and a first translation matrix corresponding to the minimum function value of the target error function, and obtain a first transfer matrix.

In some embodiments, the target parameter includes at least one of roll angle, yaw angle, pitch angle, abscissa, ordinate and altitude.

In some embodiments, using any radar in the plurality of radars as a reference radar, and based on the first transfer matrix corresponding to the reference radar, determining the second transfer matrix corresponding to other radars in the plurality of radars except the reference radar includes: for The target radar among the radars except the reference radar determines the second transfer matrix corresponding to the target radar based on the first transfer matrix corresponding to the reference radar and the inverse matrix of the first transfer matrix corresponding to the target radar.

According to the second aspect of the embodiments of this specification, there is provided a data processing device, the device comprising: an acquisition unit, configured to acquire a point cloud map of the first target scene and a point cloud obtained by scanning the first target scene by multiple radars Data; a first determination unit, configured to determine a first transfer matrix corresponding to each radar based on the obtained point cloud data corresponding to each radar and the point cloud map of the first target scene, where the first transfer matrix is the coordinates of the corresponding radar The transfer matrix of the coordinate system relative to the first target scene; the second determination unit is used to use any radar in the plurality of radars as a reference radar, based on the first transfer matrix corresponding to the reference radar, determine the difference between the plurality of radars except the reference The second transfer matrix corresponding to other radars outside the radar, the second transfer matrix is the transfer matrix of the coordinate system of the corresponding radar relative to the coordinate system of the reference radar; the data fusion unit is used for the second transfer corresponding to the reference radar and other radars The matrix performs data fusion on the point cloud data obtained by scanning the second target scene by multiple radars.

In some embodiments, when the acquiring unit is used to acquire the point cloud map of the first target scene, it is specifically configured to: based on the point cloud data collected by the preset radar located in the first target scene, determine the Based on the motion equation and observation equation of the target object equipped with the preset radar, the motion equation is used to indicate the relationship between the position of the target object and the motion data of the target object, and the observation equation is used to indicate the relationship between the preset point in the first target scene The relationship between the position and the position of the target object; based on the motion equation of the target object and the observation equation, determine the position of the target object and the positions of a plurality of preset points in the first target scene, and obtain the position of the first target scene Point cloud map.

In some embodiments, the first determination unit, when determining the first transfer matrix corresponding to each radar based on the acquired point cloud data corresponding to each radar and the point cloud map of the first target scene, includes An acquisition subunit and a determination subunit; wherein, the acquisition subunit is used to acquire a target parameter value whose target parameter of any radar satisfies a setting condition for any radar among multiple radars, and the setting condition is any radar The position of the point corresponding to the point cloud data has the largest matching degree with the position of the point in the point cloud map of the first target scene; the determination subunit is used for point cloud based on the target parameter value and the first target scene map to determine the first transfer matrix corresponding to any radar.

In some embodiments, when the obtaining subunit is used to obtain a target parameter value whose target parameter of any radar satisfies the set condition for any radar among the plurality of radars, it is specifically used to: respond to the target parameter The parameter value adjustment operation, based on the adjusted parameter value, displays the point cloud map corresponding to the point cloud data of any radar; in response to the submission operation of the parameter value of the target parameter, obtain the current parameter value of the target parameter as the target The target parameter value for the parameter.

In some embodiments, the first transfer matrix includes a first rotation matrix and a first translation matrix; the first determination unit is used to obtain point cloud data corresponding to each radar and the points of the first target scene The cloud map, when determining the first transfer matrix corresponding to each radar, is specifically used for: for any radar among multiple radars, the point cloud data corresponding to any radar and the point cloud data corresponding to the target parameters are used as the corresponding point cloud data of any radar The intermediate point cloud data, based on the intermediate point cloud data, the point cloud map of the first target scene and the target error function, determine the corresponding first rotation matrix and first translation matrix under the condition that the function value of the target error function is the smallest, and obtain first transition matrix.

In some embodiments, the second determining unit is configured to use any radar among the plurality of radars as a reference radar, and determine other radars among the plurality of radars except the reference radar based on the first transfer matrix corresponding to the reference radar When the corresponding second transfer matrix is used, it is specifically used for: for the target radar in other radars except the reference radar among the multiple radars, based on the first transfer matrix corresponding to the reference radar, and the inverse of the first transfer matrix corresponding to the target radar matrix, to determine the second transfer matrix corresponding to the target radar.

According to a third aspect of the embodiments of this specification, there is provided a terminal, including a memory, a processor, and a computer program stored in the memory and operable on the processor, wherein, when the processor executes the computer program, the above-mentioned data processing method is implemented. The action to perform.

According to a fourth aspect of the embodiments of the present specification, a computer-readable storage medium is provided. A program is stored on the computer-readable storage medium, and the program is used by a processor to execute the operations performed by the above data processing method.

According to a fifth aspect of the embodiments of the present specification, there is provided a computer program product, including a computer program, and when the program is executed by a processor, operations performed by the above data processing method are implemented.

The technical solution provided by the embodiments of this specification may include the following beneficial effects: the technical solution provided by the embodiments of this specification obtains the point cloud map of the first target scene and the point cloud data obtained by scanning the first target scene with multiple radars ; Based on the obtained point cloud data corresponding to each radar and the point cloud map of the first target scene, determine the first transfer matrix corresponding to each radar, the first transfer matrix is the coordinate system of the corresponding radar relative to the first target scene The transfer matrix of the coordinate system; using any radar in the plurality of radars as a reference radar, based on the first transfer matrix corresponding to the reference radar, determine the second transfer matrix corresponding to other radars in the plurality of radars except the reference radar, and the second transfer matrix The matrix is the transfer matrix of the coordinate system of the corresponding radar relative to the coordinate system of the reference radar, so that the data collected by each radar can be mapped to the coordinate system of the reference radar; then based on the second coordinate system corresponding to the reference radar and other radars The transfer matrix performs data fusion on the point cloud data obtained by scanning the second target scene by multiple radars, and realizes the fusion of data obtained by multiple radars.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the specification.

Description of drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the specification and together with the description serve to explain the principles of the specification.

Fig. 1 is a flow chart of a data processing method shown in this specification according to an exemplary embodiment.

Fig. 2 is a schematic diagram showing a visual display result of point cloud data according to an exemplary embodiment in this specification.

Fig. 3 is a block diagram of a data processing device shown in this specification according to an exemplary embodiment.

Fig. 4 is a schematic structural diagram of a terminal shown in this specification according to an exemplary embodiment.

Detailed ways

Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numerals in different drawings refer to the same or similar elements unless otherwise indicated. The implementations described in the following exemplary examples do not represent all implementations consistent with this specification. Rather, they are merely examples of apparatuses and methods consistent with aspects of the present specification as recited in the appended claims.

The terms used in this specification are for the purpose of describing particular embodiments only, and are not intended to limit the specification. As used in this specification and the appended claims, the singular forms "a", "the", and "the" are intended to include the plural forms as well, unless the context clearly dictates otherwise. It should also be understood that the term "and/or" as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

It should be understood that although the terms first, second, third, etc. may be used in this specification to describe various information, the information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this specification, first information may also be called second information, and similarly, second information may also be called first information. Depending on the context, the word "if" as used herein may be interpreted as "at" or "when" or "in response to a determination."

The present application provides a data processing method for processing data collected by multiple radars of an automatic driving vehicle. The data processing method can be performed by a terminal, and the terminal can be a vehicle-mounted portable terminal, for example, a mobile phone (such as a high-performance mobile phone), a tablet computer, a game console, a portable computer, a vehicle-mounted portable computer installed on an automatic driving vehicle (such as vehicle-mounted terminal), etc., the present application does not limit the specific type of the terminal.

In this application, multiple radars (for example, low-beam laser radar) are installed on the self-driving vehicle, and the detection radius of each radar can be the same or different, but the detection directions of each radar are not the same, so that the detection radius of each radar The detection ranges are all different. The self-driving vehicle collects point cloud data in the detection range corresponding to each radar through these multiple radars, and then transmits the collected point cloud data to the terminal, and the terminal performs fusion processing on the point cloud data collected by each radar , so that the fused point cloud data corresponds to the same coordinate origin and positive direction, and since the point data collected by each radar corresponds to different detection ranges, it is possible to obtain point cloud data within a larger detection range without the need for Using high-beam lidar can reduce the cost in the process of automatic driving on the basis of ensuring the detection range of the radar in the process of automatic driving.

The above is only a relevant introduction about the application scenarios of the present application. Next, the method for determining the driving route provided by the present application will be described in detail in combination with the embodiments of the present specification.

As shown in Figure 1, Figure 1 is a flow chart of a data processing method shown in this specification according to an exemplary embodiment, including the following steps: In step 101, a point cloud map of the first target scene and multiple radar scans are acquired Point cloud data obtained from the first target scene.

Wherein, the first target scene is a plant scene for factory testing of the self-driving vehicle, or the first target scene is other scenes, which are not limited in this application.

The point cloud map of the first target scene can be established in advance, and then the established point cloud map can be stored, so that the stored point cloud map of the first target scene can be directly acquired when performing radar calibration. For example, the first target scene is scanned by a preset radar in advance to obtain point cloud data of the first target scene, and then a point cloud map of the first target scene is constructed based on the point cloud data of the first target scene.

By pre-establishing the point cloud map of the first target scene, so that the point cloud map of the first target scene can be directly obtained later, without having to re-generate the point cloud map of the first target scene every time, reducing processing costs and improving processing efficiency .

Wherein, the preset radar in the first target scene is a calibrated radar, that is, the coordinate origin and positive direction corresponding to the preset radar are known. When the point cloud map of the first target scene is obtained through the preset radar, there are two ways as follows: In a possible implementation, when the point cloud data of the first target scene is obtained through the preset radar, the preset The radar is placed at any position in the first target scene, so that the first target scene is scanned (for example, laser scanning) by the preset radar placed at any position to collect the first target scene point cloud data.

In another possible implementation, the preset radar is installed on a movable object (such as a cart or trolley), so that the movable object moves in the first target scene, and during the moving process of the movable object, by installing The preset radar on the movable object scans the first target scene (for example, performs laser scanning), so as to collect point cloud data of the first target scene in real time.

When obtaining the point cloud data obtained by scanning the first target scene with multiple radars, install multiple radars on a movable object, and move the movable object in the first target scene, so that by installing on the movable object Multiple radars on the radar scan the first target scene, so as to obtain the point cloud data obtained by each radar scanning the first target scene.

When multiple radars are used to scan the first target scene to obtain point cloud data, since the coordinate origin and positive direction of each radar may be different, the coordinate systems corresponding to the point cloud data obtained by each radar are different.

Optionally, after obtaining the point cloud data of the first target scene, export the point cloud data of each radar in turn, and save the point cloud data of each radar as a point cloud data (Point Cloud Data, PCD) file, In order to send the exported PCD file to other terminals or servers, so that other terminals or servers can visually display the point cloud data based on the received PCD file.

In step 102, based on the obtained point cloud data corresponding to each radar and the point cloud map of the first target scene, determine the first transfer matrix corresponding to each radar, the first transfer matrix is the coordinate system of the corresponding radar relative to the first The transition matrix of the coordinate system of the target scene.

Wherein, for any radar in the plurality of radars, the first transfer matrix corresponding to any radar represents the point cloud data corresponding to any radar, which needs to be converted into the corresponding point cloud map part in the point cloud map of the first target scene The transformation (including rotation and translation) of the corresponding point cloud data, and because the point cloud data corresponding to any radar and the point cloud map of the first target scene correspond to the corresponding coordinate system, so that The first transfer matrix corresponding to any radar can represent the transformation through which the coordinate system of any radar is to be transformed into the coordinate system of the first target scene.

Optionally, the first transfer matrix is a 4*3 matrix, or the first transfer matrix is a 4*4 matrix, which is not limited in this application.

The following two cases are taken as examples to illustrate the composition of the first transfer matrix: if the first transfer matrix is a 4*3 matrix, the part corresponding to the 1-3 row and the 1-3 column is a rotation matrix , the part corresponding to row 4 and column 1-3 is the translation matrix.

If the first transfer matrix is a 4*4 matrix, the part corresponding to the 1-3 row and the 1-3 column is a rotation matrix, the part corresponding to the 4th row and the 1-3 column is a translation matrix, and the 1-3 column corresponds to a translation matrix. The part corresponding to the 4th row and the 4th column is a sequence (0,0,0,1), so that the first transfer matrix is a homogeneous matrix, which is convenient for subsequent matrix transformation, thereby improving the matrix transformation efficiency.

Step 103: Using any one of the multiple radars as a reference radar, based on the first transfer matrix corresponding to the reference radar, determine the second transfer matrix corresponding to other radars in the multiple radars except the reference radar, the second transfer matrix is the corresponding The transfer matrix of the coordinate system of the radar with respect to the coordinate system of the reference radar.

Since the target scene may not be involved during the driving process of the self-driving vehicle, after obtaining the first transfer matrix corresponding to each radar, by selecting one of the multiple radars as the reference radar, the The first transfer matrices corresponding to other radars other than the radar are transformed into the second transfer matrix relative to the reference radar, so that the point cloud data acquired by each radar can be fused.

Wherein, the second transfer matrix is a 4*3 matrix, or, the second transfer matrix is a 4*4 matrix, which is not limited in the present application. Optionally, dimensions of the first transfer matrix and the second transfer matrix may be the same or different.

Taking the second transfer matrix as a 4*3 matrix as an example, the same as the 4*3 transfer matrix, the part corresponding to the 1-3 rows and 1-3 columns of the second transfer matrix is a rotation matrix, and the 4th row The part corresponding to columns 1-3 is a translation matrix, and for the reference radar, the second transfer matrix of the reference radar is a zero matrix, that is, both the rotation matrix and the translation matrix of the reference radar are zero matrices.

Taking the matrix whose second transfer matrix is 4*4 as an example, it is the same as the transfer matrix of 4*4. The part corresponding to the 1-3 rows and 1-3 columns of the second transfer matrix is a rotation matrix, and the 4th row The part corresponding to the 1-3 column is the translation matrix, the part corresponding to the 1-4 row and the 4th column is the sequence (0,0,0,1), and for the reference radar, the rotation matrix of the reference radar and The translation matrices are all zero matrices.

Step 104 , based on the second transfer matrix corresponding to the reference radar and other radars, perform data fusion on the point cloud data obtained by scanning the second target scene by multiple radars.

Wherein, the second target scene is any driving scene, for example, the second target scene is the road on which the self-driving vehicle is driving, or the second target scene is other scenes, which are not limited in this application.

In a possible implementation, based on the second transfer matrix corresponding to other radars, the coordinates of each point in the point cloud data collected by each radar are transformed, so that the coordinates of each point are relative to the same coordinate system , and then obtain point cloud data corresponding to multiple points relative to the same coordinate system, thereby realizing the fusion of point cloud data.

In the embodiment of this specification, by acquiring the point cloud map of the first target scene and the point cloud data obtained by scanning the first target scene with multiple radars, based on the acquired point cloud data corresponding to each radar and the first The point cloud map of the target scene, determine the first transfer matrix corresponding to each radar, and then use any radar in the multiple radars as a reference radar, based on the first transfer matrix corresponding to the reference radar, determine the The second transfer matrix corresponding to other radars, so that the data collected by each radar can be mapped to the coordinate system of the reference radar, so that based on the second transfer matrix corresponding to the reference radar and other radars, the second scan of multiple radars The point cloud data obtained by the target scene is fused to realize the fusion of data obtained by multiple radars.

And when the first target scene is a factory building scene for factory testing of self-driving vehicles, the method provided by this application can realize the factory calibration of the radar of the self-driving vehicles, so that after the self-driving vehicles are put into use, The point cloud data of multiple radars can be fused directly based on the determined second transfer matrix.

After introducing the basic implementation process of the present application, various non-limiting implementation manners of the present application are introduced in detail below.

In some embodiments, the point cloud map of the first target scene can be acquired in the following manner.

Step 1, based on the point cloud data collected by the preset radar located in the first target scene, determine the motion equation and observation equation for the target object equipped with the preset radar.

Wherein, the motion equation is used to indicate the relationship between the position of the target object and the motion data of the target object, and the observation equation is used to indicate the relationship between the position of the preset point in the first target scene and the position of the target object.

The process of determining the equations of motion and observation equations is described below.

The process of determining the equation of motion is as follows: Optionally, the target object carries a motion sensor for acquiring motion data of the target object. For example, the motion sensor is an acceleration sensor, which is used to obtain the motion acceleration of the target object, and the motion acceleration is the motion data of the target object.

Taking the adjacent time as the first moment and the second moment as an example, based on the point cloud data collected by the preset radar at the first moment and the second moment, respectively determine the coordinates of the target object at the first moment and the second moment, Furthermore, based on the coordinates of the target object at the first moment and the second moment, the equation of motion of the target object is determined.

The above equation of motion can be referred to the following formula (1):

x _k ＝f(x _k-1 ，u _k ，w _k ) (1)

Among them, x _k represents the coordinates of the target object at the second moment, f represents the abstract function, x _k-1 represents the coordinates of the target object at the first moment, u _k represents the motion data (such as motion acceleration) of the target object, and w _k represents The noise that exists during the movement of the target object.

The determination process of the observation equation is as follows: a plurality of landmark preset points are preset in the first target scene, for example, the preset point is the position of the signboard preset in the first target scene, or, the preset Points are other types, which are not limited in this application.

If any one of the multiple preset points is detected when the target object moves to any position, the target is determined based on the current position of the target object and the position of any detected preset point The observation equation of the object.

The above observation equation can be referred to the following formula (2):

z _{k, j} = h(y _j , x _k , v _{k, j} ) (2)

Among them, z _{k, j} represents the observation of the detected preset point, h represents the abstract function, y _j represents the coordinates corresponding to the position described by the detected preset point, x _k represents the coordinates of the target object, v _k,j represent the noise existing in the observation process.

The equations of motion and observation equations involved in the above process are only two exemplary expressions. In more possible implementations, the equations of motion and observation equations can also be expressed by other formulas, which are not included in this application. limited.

Step 2: Based on the motion equation and observation equation of the target object, the position of the target object and the positions of multiple preset points in the first target scene are determined to obtain a point cloud map of the first target scene.

When constructing a point cloud map, there are two main problems to be solved, one is the positioning problem, and the other is the mapping problem. By determining the equation of motion, the positioning of the target object can be realized. By determining the observation equation, it can be realized The detection of multiple preset points in the first target scene, and the so-called map is a collection of all preset points, so by determining the position of the target object and the positions of multiple preset points in the first target scene , the construction of the point cloud map of the first target scene can be realized.

In some embodiments, based on the obtained point cloud data corresponding to each radar and the point cloud map of the first target scene, determining the first transfer matrix corresponding to each radar includes: Step 1, for any of the multiple radars Radar, to obtain the target parameter value whose target parameter of any radar satisfies the setting condition, the setting condition is the position of the point corresponding to the point cloud data of any radar, and the position of the point in the point cloud map of the first target scene the maximum degree of matching.

In a possible implementation, after obtaining the point cloud data corresponding to any radar, visual display is performed based on the obtained point cloud map of the first target scene and the point cloud data corresponding to any radar, And a parameter value adjustment control for adjusting the parameter value of the target parameter is provided on the visual interface, so that relevant technical personnel can adjust the parameter value of the target parameter by operating the parameter value adjustment control. Optionally, the parameter adjustment control is a slide bar, or the parameter adjustment control is an input box, etc., which are not limited in this application.

Wherein, the target parameter includes at least one of roll angle (Roll), yaw angle (Yaw), pitch angle (Pitch), abscissa (x), ordinate (y) and height (z).

Optionally, a target parameter corresponds to a parameter value adjustment control. Taking the target parameter including roll angle, yaw angle, pitch angle, abscissa, ordinate and altitude as an example, there are 6 parameter values provided in the visual interface Adjust the control so that the parameter values of roll angle, yaw angle, pitch angle, abscissa, ordinate and height are adjusted respectively by adjusting the control through these 6 parameter values, and then in response to the parameter value adjustment operation of the target parameter, Based on the adjusted parameter value, display the point cloud map corresponding to the point cloud data of any radar, so as to achieve the purpose of displaying the parameter adjustment result in real time based on the parameter adjustment situation, so that relevant technical personnel can know the adjustment situation of the parameter value in time Whether to meet the requirements.

Wherein, the parameter value adjustment operation on the target parameter is a trigger operation on the parameter value adjustment control. Optionally, the trigger operation may be a click operation or a drag operation, or the trigger operation may be another type of operation, which is not limited in this application.

Taking the parameter adjustment control as a slide bar as an example, if the trigger operation is a click operation, the relevant technicians can directly click anywhere on the slide bar, and the terminal will respond to the click operation on the slide bar with the corresponding The parameter value is determined as the adjusted parameter value.

Still taking the parameter value adjustment control as a slider as an example, if the trigger operation is a drag operation, the relevant technical personnel can drag the slider on the slider, and the terminal will end the drag operation in response to the drag operation on the slider. The parameter value corresponding to the position of is determined as the adjusted parameter value.

Referring to FIG. 2 , FIG. 2 is a schematic diagram showing a visual display result of point cloud data according to an exemplary embodiment in this specification. In the visual display result shown in Figure 2, the point cloud map of the first target scene is displayed, and the visual display result corresponding to the point cloud data of any radar, the visualization result corresponding to the point cloud data of any radar can be See the part shown in the rectangular box in Figure 2. In the left part of the interface as shown in Figure 2, there are also multiple slide bars (that is, parameter value adjustment controls) for adjusting the parameter values of the target parameters, which are used to adjust the roll angle and yaw respectively. Adjust the parameter values of angle, pitch angle, abscissa, ordinate and altitude.

Optionally, after the relevant technicians adjust the parameter values of each target parameter to meet the required parameter values, they trigger a submission operation in the visual interface, and the terminal responds to the submission operation of the parameter values of the target parameters to obtain the current value of the target parameter. parameter value, as the target parameter value for the target parameter.

In a possible implementation manner, a submit control is provided in the visual interface, for example, a GICP button shown in FIG. 2 , and a person skilled in the art may trigger the submit control to trigger a submit operation in the visual interface. In response to the trigger operation on the submit control, the terminal acquires the current parameter value of the target parameter as the target parameter value of the target parameter.

The foregoing is only an exemplary description when adjusting parameter values, and does not constitute a limitation to this specification. In more possible implementation manners, other methods may also be used to adjust parameter values.

Step 2: Determine a first transfer matrix corresponding to any radar based on the target parameter value and the point cloud map of the first target scene.

Wherein, the first transfer matrix includes a first rotation matrix and a first translation matrix.

In a possible implementation, for any radar among multiple radars, the point cloud data corresponding to any radar and the point cloud data corresponding to the target parameters are used as the intermediate point cloud data corresponding to any radar, based on the intermediate point The cloud data, the point cloud map of the first target scene, and the target error function determine the first rotation matrix and the first translation matrix corresponding to the minimum function value of the target error function to obtain the first transfer matrix.

Wherein, the target error function represents the error of the point cloud data corresponding to the target parameters of the intermediate point cloud data of any radar under the first transfer matrix. Based on the intermediate point cloud data, the point cloud map of the first target scene, and the target error function, determine the corresponding first rotation matrix and first translation matrix when the function value of the target error function is the smallest, that is, from the middle In the point cloud data corresponding to the point cloud data and the map of the first target scene, the nearest neighbor point (p _i , q _i ) is determined according to the target constraint condition, so that based on the nearest neighbor point (p _i , q _i ) and the formula ( 3) As shown in the target error function, the first rotation matrix and the first translation matrix are determined. Formula (3) sees the following formula:

Among them, R represents the first rotation matrix, t represents the first translation matrix, f(R, t) represents the target error function, n represents the number of nearest neighbor point pairs, p _i represents the intermediate point cloud data of any radar One point, q _i represents a point in the point cloud data corresponding to the target parameter.

For the target constraints involved in the above process, it can be the point p _i in the intermediate point cloud data of any radar, and the distance between the point cloud data corresponding to the target parameters is the smallest, that is, the target constraints can be referred to the following formula (4):

Among them, p _i represents a point in the intermediate point cloud data of any radar, q _i represents a point in the point cloud data corresponding to the target parameters, Q represents the point cloud data corresponding to the target parameters, and d(p _i , Q) represents any A distance between the point p _i in the intermediate point cloud data of the radar and the point cloud data Q corresponding to the target parameter.

By manually adjusting the parameter values of the target parameters in advance, the point cloud map corresponding to the point cloud data of each radar has been manually adjusted, and has basically matched the corresponding part of the point cloud map of the first target scene, and then obtained by manual adjustment. On the basis of the results of the first transfer matrix, the determination of the first transfer matrix can reduce the processing pressure during calculation, thereby increasing the calculation speed and reducing the calculation time.

In some embodiments, using any radar in the plurality of radars as a reference radar, based on the first transfer matrix corresponding to the reference radar, determining the second transfer matrix corresponding to other radars in the plurality of radars except the reference radar, including : For the target radar in other radars except the reference radar, based on the first transfer matrix corresponding to the reference radar and the inverse matrix of the first transfer matrix corresponding to the target radar, determine the second transfer matrix corresponding to the target radar .

In a possible implementation manner, a matrix obtained by performing dot product processing on the first transfer matrix corresponding to the reference radar and the inverse matrix of the first transfer matrix corresponding to the target radar is used as the second transfer matrix corresponding to the target radar.

Optionally, if the first transfer matrix is a 4*3 matrix, add a column (0,0,0,1) after the third column of the first transfer matrix, so that the first transfer matrix becomes a homogeneous matrix , which is convenient for matrix operations. If the first transition matrix is a 4*4 matrix, no other processing is required, and subsequent matrix calculations can be performed directly based on the first transition matrix.

Corresponding to the foregoing method embodiments, this specification also provides embodiments of a device and a terminal to which it is applied.

Referring to Fig. 3, Fig. 3 is a block diagram of a data processing device shown in this specification according to an exemplary embodiment. The data processing device includes: an acquisition unit 301, configured to acquire a point cloud map of a first target scene and a plurality of radar scans The point cloud data obtained by the first target scene; the first determination unit 302 is configured to determine the first transfer matrix corresponding to each radar based on the obtained point cloud data corresponding to each radar and the point cloud map of the first target scene , the first transfer matrix is the transfer matrix of the coordinate system of the corresponding radar relative to the coordinate system of the first target scene; the second determination unit 303 is configured to use any radar in the plurality of radars as a reference radar, based on the first target scene corresponding to the reference radar A transfer matrix, determining the second transfer matrix corresponding to other radars except the reference radar among the multiple radars, the second transfer matrix is the transfer matrix of the coordinate system of the corresponding radar relative to the coordinate system of the reference radar; the data fusion unit 304 uses Based on the second transfer matrix corresponding to the reference radar and other radars, data fusion is performed on the point cloud data obtained by scanning the second target scene by multiple radars.

In some embodiments, when the acquiring unit 301 is used to acquire the point cloud map of the first target scene, it is specifically configured to: based on the point cloud data collected by the preset radar located in the first target scene, determine The motion equation and observation equation of the target object equipped with the preset radar, the motion equation is used to indicate the relationship between the position of the target object and the motion data of the target object, and the observation equation is used to indicate the preset point in the first target scene The relationship between the position of the target object and the position of the target object; based on the motion equation of the target object and the observation equation, determine the position of the target object and the positions of a plurality of preset points in the first target scene, and obtain the first target scene point cloud map.

In some embodiments, when the first determining unit 302 is used to determine the first transfer matrix corresponding to each radar based on the acquired point cloud data corresponding to each radar and the point cloud map of the first target scene, It includes an acquisition subunit and a determination subunit; wherein, the acquisition subunit is used to acquire a target parameter value whose target parameter of any radar satisfies a setting condition for any radar among multiple radars, and the setting condition is any The position of the point corresponding to the point cloud data of the radar has the greatest matching degree with the position of the point in the point cloud map of the first target scene; the determining subunit is used to Cloud map, determine the first transfer matrix corresponding to any radar.

In some embodiments, the first transfer matrix includes a first rotation matrix and a first translation matrix; the first determination unit 302 is used to obtain point cloud data corresponding to each radar and the first target scene based on The point cloud map, when determining the first transfer matrix corresponding to each radar, is specifically used for: for any radar among multiple radars, the point cloud data corresponding to any radar and the point cloud data corresponding to the target parameters are used as any radar The corresponding intermediate point cloud data, based on the intermediate point cloud data, the point cloud map of the first target scene and the target error function, determine the first rotation matrix and the first translation matrix corresponding to the minimum function value of the target error function, Get the first transition matrix.

In some embodiments, the second determining unit 303 is configured to use any radar among the multiple radars as a reference radar, and determine the other radars among the multiple radars except the reference radar based on the first transfer matrix corresponding to the reference radar. When the second transfer matrix corresponding to the radar is used, it is specifically used for: for the target radar in other radars except the reference radar among the multiple radars, based on the first transfer matrix corresponding to the reference radar and the first transfer matrix corresponding to the target radar The inverse matrix is used to determine the second transfer matrix corresponding to the target radar.

For the implementation process of the functions and effects of each unit in the above device, please refer to the implementation process of the corresponding steps in the above method for details, and will not be repeated here.

As for the device embodiment, since it basically corresponds to the method embodiment, for related parts, please refer to the part description of the method embodiment. The device embodiments described above are only illustrative, and the modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in One place, or it can be distributed to multiple network modules. Part or all of the modules can be selected according to actual needs to achieve the purpose of the solution in this specification. It can be understood and implemented by those skilled in the art without creative effort.

The present application also provides a terminal. Referring to FIG. 4 , FIG. 4 is a schematic structural diagram of a terminal shown in this specification according to an exemplary embodiment. As shown in FIG. 4, the terminal includes a processor 410, a memory 420, and a network interface 430. The memory 420 is used to store computer instructions that can be run on the processor 410. The processor 410 is used to implement the present application when executing the computer instructions. In the data processing method provided in any embodiment, the network interface 430 is used to implement input and output functions. In more possible implementation manners, the terminal may further include other hardware, which is not limited in this application.

The present application also provides a computer-readable storage medium. The computer-readable storage medium can be in various forms. For example, in different examples, the computer-readable storage medium can be: RAM (Radom Access Memory, Random Access Memory) access memory), volatile memory, non-volatile memory, flash memory, storage drives (such as hard disk drives), solid-state drives, storage disks of any type (such as compact discs, DVDs, etc.), or similar storage media, or combinations thereof . Specifically, the computer-readable medium may also be paper or other suitable medium capable of printing programs. A computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the data processing method provided by any embodiment of the present application is implemented.

The present application also provides a computer program product, including a computer program. When the computer program is executed by a processor, the data processing method provided in any embodiment of the present application is implemented.

Those skilled in the art should understand that one or more embodiments of this specification may be provided as a method, device, terminal, computer-readable storage medium, or computer program product. Accordingly, one or more embodiments of the present description may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, one or more embodiments of the present description may employ a computer program embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein. The form of the product.

Each embodiment in this specification is described in a progressive manner, the same and similar parts of each embodiment can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the embodiment corresponding to the terminal, since it is basically similar to the method embodiment, the description is relatively simple, and for relevant parts, refer to part of the description of the method embodiment.

The foregoing describes specific embodiments of this specification. Other implementations are within the scope of the following claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. Multitasking and parallel processing are also possible or may be advantageous in certain embodiments.

Embodiments of the subject matter and functional operations described in this specification can be implemented in digital electronic circuitry, tangibly embodied computer software or firmware, computer hardware including the structures disclosed in this specification and their structural equivalents, or in A combination of one or more of . Embodiments of the subject matter described in this specification can be implemented as one or more computer programs, that is, one or more of computer program instructions encoded on a tangible, non-transitory program carrier for execution by or to control the operation of data processing apparatus. Multiple modules. Alternatively or additionally, the program instructions may be encoded on an artificially generated propagated signal, such as a machine-generated electrical, optical or electromagnetic signal, which is generated to encode and transmit information to a suitable receiver device for transmission by the data The processing means executes. A computer storage medium may be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or a combination of one or more of them.

The processes and logic flows described in this specification can be performed by one or more programmable computers executing one or more computer programs to perform corresponding functions by operating on input data and generating output. The processes and logic flows can also be performed by, and apparatus can also be implemented as, special purpose logic circuitry, such as an FPGA (Field Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit).

Computers suitable for the execution of a computer program include, for example, general and/or special purpose microprocessors, or any other type of central processing unit. Generally, a central processing unit will receive instructions and data from a read only memory and/or a random access memory. The basic components of a computer include a central processing unit for implementing or executing instructions and one or more memory devices for storing instructions and data. Typically, a computer will also include, or be operatively coupled to, one or more mass storage devices for storing data, such as magnetic or magneto-optical disks, or optical disks, to receive data therefrom or to It transmits data, or both. However, a computer is not required to have such a device. In addition, a computer may be embedded in another device such as a mobile phone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a Global Positioning System (GPS) receiver, or a device such as a Universal Serial Bus (USB) ) portable storage devices like flash drives, to name a few.

Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, for example, semiconductor memory devices (such as EPROM, EEPROM, and flash memory devices), magnetic disks (such as internal hard disks or removable disks), magneto-optical disks, and CD ROM and DVD-ROM disks. The processor and memory can be supplemented by, or incorporated in, special purpose logic circuitry.

While this specification contains many specific implementation details, these should not be construed as limitations on the scope of any inventions or of what may be claimed, but rather as primarily describing features of particular embodiments of particular inventions. Certain features that are described in this specification in multiple embodiments can also be implemented in combination in a single embodiment. On the other hand, various features that are described in a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination. Furthermore, although features may function in certain combinations as described above and even be initially so claimed, one or more features from a claimed combination may in some cases be removed from that combination and the claimed A protected combination can point to a subcombination or a variant of a subcombination.

Similarly, while operations are depicted in the figures in a particular order, this should not be construed as requiring that those operations be performed in the particular order shown, or sequentially, or that all illustrated operations be performed, to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system modules and components in the above-described embodiments should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can often be integrated together in a single software product in, or packaged into multiple software products.

Thus, certain embodiments of the subject matter have been described. Other embodiments are within the scope of the following claims. In some cases, the actions recited in the claims can be performed in a different order and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some implementations, multitasking and parallel processing may be advantageous.

Other embodiments of the description will readily occur to those skilled in the art from consideration of the specification and practice of the invention claimed herein. This description is intended to cover any modification, use or adaptation of this description. These modifications, uses or adaptations follow the general principles of this description and include common knowledge or conventional technical means in this technical field for which this description does not apply . That is, the specification is not limited to the precise structures that have been described above and shown in the drawings, and various modifications and changes may be made without departing from the scope thereof.

The above descriptions are only optional embodiments of this specification, and are not intended to limit this specification. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of this specification shall be included in this specification. within the scope of protection.

Claims

A data processing method, characterized in that the method comprises:

Obtaining a point cloud map of the first target scene and point cloud data obtained by scanning the first target scene with multiple radars;

Based on the obtained point cloud data corresponding to each radar and the point cloud map of the first target scene, determine the first transfer matrix corresponding to each radar, the first transfer matrix is the coordinate system of the corresponding radar relative to the The transition matrix of the coordinate system of the first target scene;

Using any radar among the plurality of radars as a reference radar, and based on the first transfer matrix corresponding to the reference radar, determine a second transfer matrix corresponding to other radars in the plurality of radars except the reference radar, so The second transfer matrix is a transfer matrix of the coordinate system of the corresponding radar relative to the coordinate system of the reference radar;

Based on the second transfer matrix corresponding to the reference radar and other radars, data fusion is performed on the point cloud data obtained by scanning the second target scene by the plurality of radars.
The method according to claim 1, wherein said obtaining the point cloud map of the first target scene comprises:

Based on the point cloud data collected by the preset radar located in the first target scene, determine a motion equation and an observation equation for the target object carrying the preset radar, the motion equation is used to indicate the target object The relationship between the position and the motion data of the target object, the observation equation is used to indicate the relationship between the position of the preset point in the first target scene and the position of the target object;

Based on the motion equation of the target object and the observation equation, determine the position of the target object and the positions of a plurality of preset points in the first target scene to obtain a point cloud map of the first target scene .
The method according to claim 1, wherein the first transfer matrix corresponding to each radar is determined based on the acquired point cloud data corresponding to each radar and the point cloud map of the first target scene, including :

For any radar among the plurality of radars, acquire a target parameter value whose target parameter of any radar satisfies a set condition, where the set condition is the position of a point corresponding to the point cloud data of any radar , the degree of matching with the position of the point in the point cloud map of the first target scene is the largest;

Based on the target parameter value and the point cloud map of the first target scene, a first transfer matrix corresponding to any radar is determined.
The method according to claim 3, wherein, for any radar among the plurality of radars, obtaining a target parameter value whose target parameter of any radar satisfies a set condition includes:

In response to the parameter value adjustment operation of the target parameter, based on the adjusted parameter value, display the point cloud map corresponding to the point cloud data of any radar;

In response to the submit operation of the parameter value of the target parameter, the current parameter value of the target parameter is acquired as the target parameter value of the target parameter.
The method according to claim 3, wherein the first transfer matrix comprises a first rotation matrix and a first translation matrix;

The determining the first transfer matrix corresponding to each radar based on the obtained point cloud data corresponding to each radar and the point cloud map of the first target scene includes:

For any radar among the plurality of radars, the point cloud data corresponding to the any radar and the point cloud data corresponding to the target parameters are used as the intermediate point cloud data corresponding to the any radar, based on the intermediate The point cloud data, the point cloud map of the first target scene, and the target error function, determine the corresponding first rotation matrix and first translation matrix under the condition that the function value of the target error function is the smallest, and obtain the first transfer matrix.
The method according to claim 3, wherein the target parameters include at least one of roll angle, yaw angle, pitch angle, abscissa, ordinate and height.
The method according to claim 1, characterized in that, using any radar in the plurality of radars as a reference radar, based on the first transfer matrix corresponding to the reference radar, determine The second transfer matrix corresponding to other radars other than the reference radar includes:

For the target radars among the radars except the reference radar, based on the first transfer matrix corresponding to the reference radar and the inverse matrix of the first transfer matrix corresponding to the target radar, determine the The second transfer matrix corresponding to the target radar.
A radar calibration device, characterized in that the device comprises:

An acquisition unit, configured to acquire a point cloud map of the first target scene and point cloud data obtained by scanning the first target scene with multiple radars;

The first determination unit is configured to determine the first transfer matrix corresponding to each radar based on the acquired point cloud data corresponding to each radar and the point cloud map of the first target scene, the first transfer matrix is the corresponding radar The coordinate system of the coordinate system relative to the transfer matrix of the coordinate system of the first target scene;

The second determining unit is configured to use any radar among the plurality of radars as a reference radar, and determine the correspondence of other radars among the plurality of radars except the reference radar based on the first transfer matrix corresponding to the reference radar. The second transfer matrix, the second transfer matrix is the transfer matrix of the coordinate system of the corresponding radar relative to the coordinate system of the reference radar;

The data fusion unit is configured to perform data fusion on the point cloud data obtained by scanning the second target scene by the plurality of radars based on the second transfer matrix corresponding to the reference radar and other radars.
A terminal, characterized in that the terminal includes a memory, a processor, and a computer program stored in the memory and operable on the processor, wherein the processor implements the computer program according to claims 1 to 7 when executing the program. The operations performed by any one of the data processing methods.
A computer-readable storage medium, characterized in that a program is stored on the computer-readable storage medium, and the program is executed by a processor executing the data processing method according to any one of claims 1 to 7 operate.