CN117670785A - Ghost detection method for point cloud maps - Google Patents

Abstract

The application discloses a ghost detection method of a point cloud map, and belongs to the technical field of electronic information. The method comprises the following steps: dividing a point cloud map to be detected into a plurality of grids; determining a query key frame from a local point cloud track included in a target grid; determining at least one matching frame of the query key frame in the multi-frame point cloud track; determining registration results of the query key frame and each matching frame of the query key frame; in response to the presence of a registration result satisfying the requirements, the target grid is determined to be a grid in which ghosts are present. In the method, the full coverage of ghost detection in the point cloud map is realized by dividing the point cloud map into grids for detection, and the detection granularity is finer. And selecting the query key frame and the matching frame for registration, and improving the detection accuracy, so that an accurate high-precision map can be constructed based on the point cloud map after ghost detection, and the normal running of the automatic driving vehicle is ensured based on the accurate high-precision map. Since specific elements are not required to be extracted, the restriction of the scene on ghost detection is reduced.

Description

Ghost detection method for point cloud maps

Technical field

Embodiments of the present application relate to the field of electronic information technology, and in particular to a ghost detection method for point cloud maps.

Background technique

At present, electronic high-precision maps have become an indispensable part of many travel scenarios, especially in the field of autonomous driving. Electronic high-precision maps are an important prerequisite for ensuring the normal driving of autonomous vehicles, and the construction of point cloud maps is an important part of electronic high-precision maps. An essential link in the production process. Point cloud is a collection of dispersed points with accurate angle and distance information that is presented through radar collection of surrounding environment object information. The map constructed using point cloud is called a point cloud map. The quality of the point cloud map determines the accuracy of the electronic high-precision map, and whether there are ghosts in the point cloud map determines the quality of the point cloud map. Therefore, ghost detection needs to be performed on point cloud maps.

Contents of the invention

Embodiments of the present application provide a ghost detection method, device, equipment and storage medium for point cloud maps, which can be used to improve detection accuracy and reduce scene restrictions on ghost detection. The technical solution is as follows:

On the one hand, embodiments of the present application provide a ghost detection method for point cloud maps. The method includes:

Divide the point cloud map to be detected into multiple grids. The point cloud map includes multi-frame point cloud trajectories, and each grid includes local point cloud trajectories in the multi-frame point cloud trajectories;

Determine query keyframes from local point cloud trajectories included in the target mesh;

Determine at least one matching frame of the query key frame in the multi-frame point cloud trajectory, where the matching frame is the local point cloud trajectory within the reference range where the query key frame is located;

Determine the registration results of the query key frame and each matching frame of the query key frame;

In response to the existence of a registration result that satisfies the requirements, the target grid is determined to be a grid in which ghosting exists.

In one possible implementation, the query keyframe is determined from the local point cloud trajectory included in the target mesh, including:

Determine the number of road layers in the target grid;

A reference number of point cloud trajectories are determined from the local point cloud trajectories included in the target grid as query key frames, and the reference number is determined based on the number of road layers.

In a possible implementation, determining at least one matching frame of the query key frame in the multi-frame point cloud trajectory includes:

Filter out local point cloud trajectories within the reference range from multi-frame point cloud trajectories;

In response to multiple local point cloud trajectories being included in the reference range, the local point cloud trajectories that satisfy the time interval among the multiple local point cloud trajectories are determined as matching frames of the query key frame.

In a possible implementation, determining the registration results of the query key frame and each matching frame of the query key frame includes:

Determine the subgraph of the query key frame, which is obtained based on the point cloud trajectory included in the query key frame;

For the query key frame and any matching frame, determine the sub-image of any matching frame, and the sub-image of any matching frame is obtained based on the point cloud trajectory included in any matching frame;

Multiple pose transformation matrices are determined based on the subgraph of the query key frame and the subgraph of any matching frame. Any pose transformation matrix includes rotation transformation parameters and translation transformation parameters that transform the query key frame into any matching frame. at least one;

Calculate the mean value of the pose transformation matrix that meets the requirements among multiple pose transformation matrices, and use the calculated result as the registration result between the query key frame and any matching frame.

In a possible implementation, determining the subgraph of the query key frame includes:

Merge the point cloud trajectories included in the query key frame and the relevant point cloud trajectories of the query key frame to obtain a subgraph of the query key frame. The relevant point cloud trajectories of the query key frame include the timestamps adjacent to the query key frame within the target grid. point cloud trajectory.

In a possible implementation, determining the subgraph of any matching frame includes:

Merge the point cloud trajectory included in any matching frame with the relevant point cloud trajectory of any matching frame to obtain a sub-image of any matching frame. The relevant point cloud trajectory of any matching frame includes the grid where any matching frame is located. Point cloud trajectories adjacent to any matching frame timestamp within .

In a possible implementation, multiple pose transformation matrices are determined based on the subgraph of the query key frame and the subgraph of any matching frame, including:

Determine multiple sets of local point cloud data corresponding to the sub-image of the query key frame and the global point cloud data corresponding to the sub-image of any matching frame;

Register multiple sets of local point cloud data with global point cloud data respectively to obtain multiple pose transformation matrices. One set of local point cloud data corresponds to one pose transformation matrix, and any set of local point cloud data corresponds to the pose transformation matrix. The matrix includes at least one of a rotation transformation parameter and a translation transformation parameter that transforms the query key frame into any matching frame through any set of local point cloud data.

In a possible implementation, multiple sets of local point cloud data corresponding to the subgraph of the query key frame are determined, including:

In the global point cloud data included in the sub-picture of the query key frame, the point cloud data that meets the proportion requirements are filtered out according to different areas, and the point cloud data that meets the proportion requirements filtered out in each area are used as the corresponding point cloud data of the sub-picture of the query key frame. A set of local point cloud data.

In a possible implementation, after determining the registration results of the query key frame and each matching frame of the query key frame, it also includes:

For any matching frame, the pose error of the query key frame is calculated based on the registration result and the registration transformation matrix of the query key frame and any matching frame. The registration transformation matrix includes converting the query key frame through the global point cloud data of the query key frame. The frame is transformed to at least one of a rotation transformation parameter and a translation transformation parameter of any matching frame;

In response to the pose error being greater than the pose error threshold, it is determined that the registration result between any matching frame and the query key frame meets the requirements.

In a possible implementation, after determining the target mesh as a mesh with ghosts, the following steps are also included:

Merge the meshes with ghosts and determine the ghosting area based on the merged meshes;

Repair the point cloud trajectories included in the ghosted area.

On the other hand, a ghost detection device for point cloud maps is provided, and the device includes:

The division module is used to divide the point cloud map to be detected into multiple grids. The point cloud map includes multi-frame point cloud trajectories, and each grid includes local point cloud trajectories in the multi-frame point cloud trajectories;

A determination module for determining query key frames from the local point cloud trajectory included in the target grid;

The determination module is also used to determine at least one matching frame of the query key frame in the multi-frame point cloud trajectory, where the matching frame is the local point cloud trajectory within the reference range where the query key frame is located;

The determination module is also used to determine the registration results of the query key frame and each matching frame of the query key frame;

The determining module is also configured to determine the target grid as a grid with ghosting in response to the existence of a registration result that meets the requirements.

In a possible implementation, the determination module is used to determine the number of road layers in the target grid; determine a reference number of point cloud trajectories as query key frames from the local point cloud trajectories included in the target grid, and the reference number is based on The number of road layers is determined.

In a possible implementation, the device further includes: a screening module, used to filter out local point cloud trajectories within the reference range from multi-frame point cloud trajectories;

A determination module, configured to determine, in response to the reference range including multiple local point cloud trajectories, the local point cloud trajectories that satisfy the time interval among the multiple local point cloud trajectories as matching frames of the query key frame.

In a possible implementation, the determination module is used to determine the subgraph of the query key frame. The subgraph of the query key frame is obtained based on the point cloud trajectory included in the query key frame; for the query key frame and any matching frame, determine The subgraph of any matching frame is obtained based on the point cloud trajectory included in any matching frame; multiple pose transformation matrices are determined based on the subgraph of the query key frame and the subgraph of any matching frame, Any pose transformation matrix includes at least one of a rotation transformation parameter and a translation transformation parameter that transforms the query key frame into any matching frame;

The device also includes: a calculation module for averaging the pose transformation matrices that meet the requirements among the multiple pose transformation matrices, and using the calculated result as a registration result between the query key frame and any matching frame.

In a possible implementation, the device further includes: a merging module for merging the point cloud trajectories included in the query key frame and the relevant point cloud trajectories of the query key frame to obtain a subgraph of the query key frame, and the query key frame The relevant point cloud trajectories include point cloud trajectories adjacent to the query keyframe timestamp within the target grid.

In a possible implementation, the merging module is used to merge the point cloud trajectory included in any matching frame with the relevant point cloud trajectory of any matching frame to obtain a sub-image of any matching frame. The relevant point cloud trajectories include point cloud trajectories adjacent to the timestamp of any matching frame within the grid where any matching frame is located.

In one possible implementation, the determination module is used to determine multiple sets of local point cloud data corresponding to the sub-image of the query key frame and global point cloud data corresponding to the sub-image of any matching frame;

The device also includes: a registration module, used to register multiple sets of local point cloud data with global point cloud data to obtain multiple pose transformation matrices. A set of local point cloud data corresponds to a pose transformation matrix. Any one The pose transformation matrix corresponding to the set of local point cloud data includes at least one of a rotation transformation parameter and a translation transformation parameter that transforms the query key frame into any matching frame through any set of local point cloud data.

In a possible implementation, the filtering module is used to filter out the point cloud data that meets the proportion requirements according to different regions from the global point cloud data included in the sub-graph of the query key frame, and filter out the point cloud data that meets the proportion requirements in each region. The required point cloud data is used as a set of local point cloud data corresponding to the subgraph of the query key frame.

In a possible implementation, the calculation module is also used to calculate, for any matching frame, the pose error of the query key frame based on the registration result and the registration transformation matrix between the query key frame and any matching frame. The transformation matrix includes at least one of a rotation transformation parameter and a translation transformation parameter that transforms the query key frame into any matching frame through the global point cloud data of the query key frame;

The determination module is also used to determine that the registration result of any matching frame and the query key frame meets the requirements in response to the pose error being greater than the pose error threshold.

In one possible implementation, the determination module is also used to merge meshes with ghosts, and determine the ghost area based on the merged meshes;

The device also includes: a repair module for repairing the point cloud trajectory included in the ghost area.

On the other hand, a computer device is provided. The computer device includes a processor and a memory. At least one computer program is stored in the memory. The at least one computer program is loaded and executed by the processor, so that the computer device implements any of the above point clouds. Ghost detection method for maps.

On the other hand, a computer-readable storage medium is also provided. At least one computer program is stored in the computer-readable storage medium. The at least one computer program is loaded and executed by the processor to enable the computer to implement any of the above point cloud maps. Ghost detection method.

On the other hand, a computer program product or computer program is also provided, the computer program product or computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs any of the above ghost detection methods of the point cloud map.

The technical solutions provided by the embodiments of this application at least bring the following beneficial effects:

By dividing the point cloud map into grids and converting the large-scale point cloud map into point cloud trajectories in multiple small-scale grids, full coverage of ghost detection in the point cloud map is achieved, and the detection granularity is finer. Select the query key frame and the matching frame, and improve the detection accuracy by registering the query key frame and the matching frame, so that an accurate high-precision map can be constructed based on the point cloud map after ghost detection, and then based on the accurate high-precision map The map ensures the normal driving of autonomous vehicles. Since there is no need to extract specific elements, it reduces the scene's restrictions on ghost detection.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings without exerting creative efforts.

Figure 1 is a schematic diagram of an implementation environment provided by an embodiment of the present application;

Figure 2 is a flow chart of a ghost detection method for point cloud maps provided by an embodiment of the present application;

Figure 3 is a schematic diagram of a point cloud map grid division provided by an embodiment of the present application;

Figure 4 is a schematic diagram of generating central local point cloud data provided by an embodiment of the present application;

Figure 5 is a schematic diagram of generating local point cloud data of an outer contour provided by an embodiment of the present application;

Figure 6 is a schematic diagram of generating upper and lower local point cloud data provided by an embodiment of the present application;

Figure 7 is a schematic diagram of generating left and right local point cloud data provided by an embodiment of the present application;

Figure 8 is a schematic diagram of generating left diagonal local point cloud data provided by an embodiment of the present application;

Figure 9 is a schematic diagram of generating right diagonal local point cloud data provided by an embodiment of the present application;

Figure 10 is a schematic diagram of a point cloud map after determining the presence of ghosts in the grid provided by the embodiment of the present application;

Figure 11 is a schematic diagram of a ghost detection device for point cloud maps provided by an embodiment of the present application;

Figure 12 is a schematic structural diagram of a server provided by an embodiment of the present application;

Figure 13 is a schematic structural diagram of a ghost detection device for point cloud maps provided by an embodiment of the present application.

Detailed ways

In order to make the purpose, technical solutions and advantages of the present application clearer, the embodiments of the present application will be further described in detail below with reference to the accompanying drawings.

The embodiment of the present application provides a method for detecting ghosts in point cloud maps. Please refer to Figure 1 , which shows a schematic diagram of the implementation environment of the method provided by the embodiment of the present application. The implementation environment may include: a terminal 11 and a server 12 .

Among them, the terminal 11 is installed with an application or web page capable of detecting ghosting. When the application or web page needs to detect ghosting, the method provided by the embodiment of the present application can be used for display. The server 12 can store the point cloud map that needs to be detected, and the terminal 11 can obtain the point cloud map that needs to be detected from the server 12 . Of course, the acquired point cloud map that needs to be detected can also be stored on the terminal 11 .

Alternatively, the terminal 11 may be a smart device such as a mobile phone, a tablet computer, a personal computer, etc. The server 12 may be one server, a server cluster composed of multiple servers, or a cloud computing service center. The terminal 11 and the server 12 establish a communication connection through a wired or wireless network.

Optionally, the terminal 11 can be any electronic product that can perform human-computer interaction with the user through one or more methods such as keyboard, touch pad, touch screen, remote control, voice interaction or handwriting device, such as PC (Personal Computer, Personal computers), mobile phones, smartphones, PDA (Personal Digital Assistant, personal digital assistant), wearable devices, PPC (Pocket PC, handheld computer), tablet computers, smart cars, smart TVs, smart speakers, etc. The server 12 may be one server, a server cluster composed of multiple servers, or a cloud computing service center. The terminal 11 and the server 12 establish a communication connection through a wired or wireless network.

Those skilled in the art should understand that the above-mentioned terminal 11 and server 12 are only examples. If other existing or possible terminals or servers that may appear in the future are applicable to this application, they should also be included in the protection scope of this application, and are hereby referred to as References are included here.

An embodiment of the present application provides a method for detecting ghosts in point cloud maps. The flow chart of the method for detecting ghosts in point cloud maps is shown in Figure 2. This method can be implemented based on the implementation environment shown in Figure 1 above. The ghost detection method of the point cloud map can be executed by the terminal or the server, or can also be implemented by the interaction between the terminal and the server. This embodiment of the present application takes the server executing the method as an example to illustrate. Referring to Figure 2, the method includes steps 201 to 205.

Step 201: Divide the point cloud map to be detected into multiple grids.

In the embodiment of this application, before ghost detection is performed on the map, the point cloud map is first obtained. The point cloud map includes multi-frame point cloud trajectories, and each grid includes local point cloud trajectories in the multi-frame point cloud trajectories. The embodiments of the present application do not limit the acquisition method of the point cloud map. For example, the area to be constructed is scanned multiple times through radar. Each time a scan is completed, a frame of point cloud trajectory is output. Multiple frame point cloud trajectories constitute the point cloud of the area. map.

When dividing the point cloud map into meshes, this includes but is not limited to determining the number and size of the meshes based on the range spanned by the point cloud map and the required accuracy of the point cloud map. Divide the point cloud map into a number of grids according to the determined number and size. Each grid includes all local point cloud trajectories within the grid area. The number of local point cloud trajectories included in each grid is at least one. The number of local point cloud trajectories contained in each grid can be the same or different. .

By dividing the point cloud map to be detected into multiple grids, the large-scale point cloud map can be reduced into multiple small-scale grids, and ghost detection can be performed on the small-scale point cloud trajectories included in the grid. , the detection granularity is finer, which can improve the accuracy of ghost detection and achieve full coverage of ghost detection.

Taking the point cloud map grid division diagram shown in Figure 3 as an example, all point cloud trajectories in the figure constitute the point cloud map of the area. Within the scope of the point cloud map, the point cloud map is divided into multiple grids according to the length H meters and the width W meters.

Optionally, after the radar completes a scan and outputs a frame of point cloud trajectory, the point cloud trajectory of each frame can be optimized through a mapping optimization algorithm to improve the accuracy of the point cloud pose in each frame of point cloud trajectory. Among them, the mapping optimization algorithm refers to an algorithm that removes the deviation of the point cloud pose and obtains an accurate point cloud pose. Point cloud pose is used to express the transformation relationship between the world coordinate system and the camera coordinate system, including two transformation parameters, rotation and translation, usually represented by a matrix. The world coordinate system is a coordinate system defined before scanning and serves as a reference when scanning and building trajectories. The camera coordinate system refers to the coordinate system constructed with the radar as the origin. Since the radar moves during the scanning process, the camera coordinate system also changes with the movement of the radar. Therefore, the coordinates of the same frame point cloud in the world coordinate system are the same as those of the camera. The coordinates in the coordinate system are different, and there are corresponding transformation relationships.

Step 202: Determine the query key frame from the local point cloud trajectory included in the target grid.

The query key frame is any point cloud trajectory among the local point cloud trajectories included in the target grid, and the target grid is any grid among multiple grids. In the embodiment of this application, the number of query key frames in each grid is not necessarily the same. The number of query key frames in the grid can be determined based on the local point cloud trajectories included in the grid and the number required for detection. . Optionally, the method of determining the query key frame includes but is not limited to steps 2021-2022.

Step 2021: Determine the number of road layers in the target grid.

The embodiments of the present application do not limit the method of determining the number of road layers in the target grid, including but not limited to determining the number of road layers L contained in the area based on the radar scanning area corresponding to the point cloud trajectory included in the target grid. For example, if there is a three-story viaduct and a common two-way multi-lane in the radar scanning area corresponding to the point cloud trajectory included in a target grid, then the number of road layers L in the target grid is determined to be 3.

Step 2022: Determine a reference number of point cloud trajectories as query key frames from the local point cloud trajectories included in the target grid, and the reference number is determined based on the number of road layers.

In one possible implementation, determining a reference number of point cloud trajectories as query key frames from the local point cloud trajectories included in the target grid includes: based on the number of road layers L in the target grid, within the target grid Select L*K frame point cloud trajectories from all included point cloud trajectories as L*K query key frames of the target grid. Among them, K can be an adjustment coefficient, for example, selected according to the complexity of the point cloud trajectory in the grid.

Optionally, if there is no road in the radar scanning area corresponding to the point cloud trajectory included in the target grid, K frames of point cloud trajectories can also be directly selected as the K query key frames of the target grid.

Step 203: Determine at least one matching frame of the query key frame in the multi-frame point cloud trajectory. The matching frame is a local point cloud trajectory within the reference range where the query key frame is located.

The embodiment of the present application does not limit the method of determining the number of matching frames of the query key frame. For example, it can be determined based on the number of point clouds contained in the query key frame, or it can also be determined by the size of the target grid. Optionally, the method of determining at least one matching frame includes but is not limited to steps 2031-2032.

Step 2031: Filter out the local point cloud trajectories within the reference range from the multi-frame point cloud trajectories.

The embodiment of the present application does not limit the reference range. The reference range may be larger than the area of the target grid or smaller than the area of the target grid. For example, the first value is set according to the standard that can ensure the establishment of association between multiple frame trajectories, with the target query key frame as the center of the circle, the first value as the radius, and the reference range for filtering the local point cloud trajectories is determined.

When filtering out the local point cloud trajectories within the reference range from the multi-frame point cloud trajectories, you can filter the local point cloud trajectories near the query key frame in the geometric space of the radar scanning area according to the determined reference range, and filter out the local point cloud trajectories. The point cloud trajectory can be the local point cloud trajectory in the target grid except the query key frame, or it can be the local point cloud trajectory in the adjacent grid of the target grid.

For example, set the first value to 50 meters, use the first value as the radius, and the query key frame as the center of the circle to determine the reference range for filtering local point cloud trajectories near the query key frame. This radius can cover common two-way multi-lane scenarios, And it can establish the correlation between multiple frame trajectories, making the coverage of ghost detection more complete.

Step 2032: In response to the reference range including multiple local point cloud trajectories, determine the local point cloud trajectories that satisfy the time interval among the multiple local point cloud trajectories as matching frames of the query key frame.

Optionally, before determining the local point cloud trajectory that satisfies the time interval, set a time interval threshold T for selecting matching frames. If among all the local point cloud trajectories filtered out, the time interval between any two local point cloud trajectories is less than T, the point clouds contained in the two local point cloud trajectories are considered to be similar, so when selecting the matching frame of the query key frame, Select any one of the two local point cloud trajectories as the matching frame of the query key frame. If the time interval between any two local point cloud trajectories among all filtered local point cloud trajectories is greater than T, the point clouds contained in all local point cloud trajectories are considered to be dissimilar, and all filtered local point cloud trajectories can be used as Query the matching frames of key frames.

Regardless of whether the time interval between any two local point cloud trajectories is less than T, among the local point cloud trajectories near the filtered query key frame, at least one local point cloud trajectory that satisfies the time interval is selected as the matching frame of the query key frame.

For example, within the reference range of the query key frame q, three local point cloud trajectories, namely local point cloud trajectory o, local point cloud trajectory s, and local point cloud trajectory p, are sequentially filtered out. Among them, local point cloud trajectory o is filtered out. The time interval between the local point cloud trajectory s and the local point cloud trajectory s is less than T. If the time interval between the local point cloud trajectory o and the local point cloud trajectory p is greater than T, then the local point cloud trajectory s is removed and the local point cloud trajectory o and the local point cloud trajectory are selected. p serves as the two matching frames of the query key frame q.

Step 204: Determine the registration results of the query key frame and each matching frame of the query key frame.

After determining at least one matching frame of the query key frame, the registration result of the query key frame and each matching frame needs to be determined to reflect the matching degree between the query key frame and each matching frame. The registration result between the query key frame and any matching frame refers to a matrix that can reflect the transformation relationship between the query key frame and any matching frame, including at least one of a translation transformation parameter and a rotation change parameter. Optionally, the method of determining the registration result includes but is not limited to steps 2041-2044.

Step 2041: Determine the subgraph of the query key frame. The subgraph of the query key frame is obtained based on the point cloud trajectory included in the query key frame.

In one possible implementation, the point cloud trajectories included in the query key frame and the relevant point cloud trajectories of the query key frame are merged to obtain a subgraph of the query key frame. The relevant point cloud trajectories of the query key frame include the target grid. Point cloud trajectories adjacent to the query keyframe timestamp.

Among them, the timestamp can refer to the total number of seconds from 00:00:00 on January 1, 1970 Greenwich Time (08:00:00 on January 1, 1970, Beijing time) to the present. By using numbers Signature technology generates data, and the objects of the signature include original file information, signature parameters, signature time and other information.

The query keyframe and the local point cloud trajectories adjacent to the query keyframe timestamp in the target grid are merged to obtain the query keyframe subgraph. Compared with the query keyframe of a single frame, the sub-image point cloud of the generated query keyframe is dense, the spatial structure is complete, the scope is wider, and the common view area is more, which is conducive to the construction of line and surface constraints.

Exemplarily, the query key frame q and the local point cloud trajectories adjacent to its timestamp in the target grid are merged to generate a subgraph S _q of the query key frame.

Step 2042: For any matching frame of the query key frame, determine the sub-image of any matching frame. The sub-image of any matching frame is obtained based on the point cloud trajectory included in any matching frame.

In a possible implementation, consistent with the above method of obtaining the subgraph of the query key frame, the point cloud trajectory included in any matching frame is merged with the relevant point cloud trajectory of any matching frame to obtain any matching frame The subgraph of , the relevant point cloud trajectory of any matching frame includes the point cloud trajectory adjacent to the timestamp of any matching frame within the grid where any matching frame is located.

For example, the matching frame p and the local point cloud trajectories adjacent to its timestamp in the grid where the matching frame is located are merged to generate a subgraph S _p of the matching frame.

Step 2043: Determine multiple pose transformation matrices based on the sub-image of the query key frame and the sub-image of any matching frame. Any pose transformation matrix includes rotation transformation parameters and translation transformations that transform the query key frame into any matching frame. at least one of the parameters.

In a possible implementation, the pose transformation matrix is determined based on multiple sets of local point cloud data corresponding to the sub-image of the query key frame and global point cloud data corresponding to the sub-image of any matching frame. This application does not determine the query key. Multiple sets of local point cloud data corresponding to the sub-images of the frame and global point cloud data corresponding to any sub-image of the matching frame are defined.

Among them, the multiple sets of local point cloud data corresponding to the sub-image of the query key frame refer to the point cloud data in any one or any multiple areas that meet the proportion requirements in the global point cloud data included in the sub-image of the query key frame. The point cloud The data includes but is not limited to the number of point clouds and the pose of each point in these point clouds. For example, the scale requirement may be a scale threshold set in advance, determined according to the required map accuracy. Correspondingly, the global point cloud data corresponding to the sub-image of any matching frame includes but is not limited to the number of global point clouds of the matching frame and the pose of each point in these point clouds.

In a possible implementation, when determining multiple sets of local point cloud data corresponding to the sub-image of the query key frame, the global point cloud data included in the sub-image of the query key frame can be filtered out according to different areas to meet the proportion requirements. The point cloud data that meets the proportion requirements are filtered out in each area as a set of local point cloud data corresponding to the subgraph of the query key frame.

After generating a sub-image of the query key frame and a sub-image of the matching frame respectively for the query key frame and any matching frame, at least two sets of local point cloud data are generated for the sub-image of the query key frame, and each set of local point cloud data corresponds to Query different regions of a subgraph for keyframes. Each set of local point cloud data generated corresponds to at least one area of the subgraph of the query key frame. Different areas correspond to different generation methods, including but not limited to center, outer contour, top and bottom, left and right, left diagonal, and right diagonal. way of generating. When choosing a method to generate local point cloud data, it is necessary to ensure that the point cloud distribution in the generated local point cloud data is relatively uniform to prevent the lack of point clouds in a certain dimension from causing excessive deviations in pose solutions. Using local point cloud data for registration can avoid reducing the weight of other areas because a certain area in the point cloud trajectory is rich in structure and dominates the registration, resulting in incorrect registration results.

Take the schematic diagram of generating central local point cloud data shown in Figure 4 as an example. The point clouds selected by the dotted line in the middle of Figure 4 are a set of central local point clouds generated based on the subgraph of the query key frame. These point clouds are The data contained is the local point cloud data in the center of the group. Referring to Figures 5 to 9, the results of generating local point cloud data for the subgraph of the query key frame using the outer contour, upper and lower, left and right, left diagonal, and right diagonal generation methods are shown respectively. Figure 5-Fig. The point cloud data selected by the dotted line in 9 is the outer contour, upper and lower, left and right, diagonal left and diagonal right local point cloud data generated based on the key sub-image frame of the query.

Exemplarily, after generating at least two sets of local point cloud data of the subgraph of the query key frame according to different areas, that is, different generation methods, multiple sets of local point cloud data that meet the proportion requirements are filtered out. Optionally, before filtering out multiple sets of local point cloud data that meet the scale requirement, set the overlapping area scale threshold of any set of local point cloud data and the submap of the matching frame as the scale requirement according to the required map accuracy.

In one possible implementation, before filtering multiple sets of local point cloud data that meet the scale requirements, it is necessary to calculate each set of local point cloud data of the sub-image of the query key frame and the sub-image of any matching frame of the query key frame. proportion of overlapping areas. The method of calculating the proportion of overlapping areas includes but is not limited to the following steps 1 to 3.

Step 1: Determine each point in any set of local point cloud data corresponding to the subgraph of the query key frame, and calculate each point in the target local point cloud data based on the registration transformation matrix corresponding to the query key frame and any matching frame. The projection point of the point in the above matching frame subgraph. Among them, the registration transformation matrix refers to the transformation relationship between the query key frame and each matching frame calculated through the mapping optimization algorithm after determining the query key frame and each matching frame, including the parameters of rotation transformation and translation transformation. at least one of them. The relationship between the query key frame and each matching frame is not necessarily the same. Therefore, the query key frame and any matching frame have corresponding registration transformation matrices.

Step 2: Determine the nearest neighbor point of each projection point in the global point cloud data of the above matching frame subgraph. Calculate the spatial distance between each projection point and its nearest neighbor point, compare each spatial distance with the set spatial distance threshold, and obtain all spatial distances smaller than the spatial distance threshold through filtering. If any calculated spatial distance is greater than the set spatial distance threshold, it is considered that there is no correspondence between the nearest neighbor point corresponding to the spatial distance and the projection point corresponding to the nearest neighbor point, and the spatial distance is discarded.

Step 3: Determine the number of filtered spatial distances that are smaller than the spatial distance threshold, and determine the ratio of the number of filtered spatial distances to the number of points in the set of local point cloud data as the set of local point cloud data and The proportion of the overlapping area of the matching frame subgraphs.

Illustratively, after the above-mentioned matching relationship between the query key frame q and the matching frame p is determined, a mapping optimization algorithm is used to determine the registration transformation matrix T _pq from the query key frame q to the matching frame p. According to the sub-image S _q of the query key frame, four local point cloud data are generated in the form of center, upper and lower, left and right, and outer contour. Determine the central local point cloud data S _q1 of the sub-image S _q of the query key frame. There are 4 points in the matching frame, namely A ₁ , A ₂ , A ₃ , and A ₄ . Calculate the position of each point in the matching frame according to A'=TpqA. Projection points A ₁ ', A ₂ ', A ₃ ', A ₄ ' in the subgraph S _p .

For example, after completing the above calculation of all projection points in the matching frame sub-image _Sp , it is determined that the nearest neighbor point of A ₁ ' in the matching frame sub-image _Sp is B ₁ and the nearest neighbor point of A ₂ ' is B _2. The nearest neighbor point of A ₃ ' is B ₃ and the nearest neighbor point of A ₄ ' is B ₄ . Calculate the spatial distance between each projection point and the corresponding nearest neighbor point as d ₁ , d ₂ and d ₃ respectively. , _d4 . Compare each calculated spatial distance with the set spatial distance threshold, and discard d ₄ that is greater than the spatial distance threshold. Determine the number of filtered spatial distances to be 3, and the number of point clouds in this set of local point cloud data to be 4. The proportion of the overlapping area between the central local point cloud data S _q1 and the matching frame sub-image S _p is obtained as

Optionally, after determining the overlapping area ratio between all local point cloud data of the sub-image of the query key frame and the sub-image of the matching frame, filter out the local point cloud data that meets the proportion requirements and determine it as the sub-image of the query key frame. Corresponding multiple sets of local point cloud data.

In a possible implementation, if the calculated proportion of the overlapping area between any set of local point cloud data and the matching frame sub-image is greater than the proportion requirement, then the set of local point cloud data and the matching frame sub-image are considered to be registered. The calculated results have credibility; if the calculated ratio of the overlapping area between the local point cloud data and the matching frame sub-image is less than the proportion threshold, it is considered that the local point cloud data is registered with the matching frame sub-image due to the small overlapping area. The calculated results are unreliable. Therefore, the local point cloud data of each group of local point cloud data and the matching frame sub-image whose overlapping area ratio is greater than the ratio requirement are used as a corresponding set of local point cloud data for the sub-image of the query key frame. The ratio of all overlapping areas is The local point cloud data that is larger than the proportion requirement is determined as multiple sets of local point cloud data corresponding to the sub-image of the query key frame.

For example, the above steps of determining the proportion of the overlapping area between the local point cloud data and the matching frame sub-image need to be repeated four times, that is, the four local point cloud data generated by querying the key frame sub-image S _q need to calculate the local point cloud data and The proportion of overlapping areas of matching frame subgraphs. After calculation, the proportion of the overlapping area between the three sets of local point cloud data S _q1 of the center local point cloud data S _q1 , the upper and lower local point cloud data S _q2 , and the left and right local point cloud data S q3 and the matching frame sub-image S _p is greater than the proportion requirement, and it is determined that These three sets of local point cloud data are three sets of local point cloud data corresponding to the sub-image S _q of the query key frame. The overlapping area of the outer contour local point cloud data S _q4 and the matching frame sub-image S _p is smaller than the overlap threshold, so the outer contour local point cloud data S _q4 is discarded.

For the multiple sets of local point cloud data corresponding to the sub-image of the query key frame determined above and the global point cloud data of the sub-image of any matching frame, register the multiple sets of local point cloud data with the global point cloud data respectively, and obtain Multiple pose transformation matrices. A set of local point cloud data corresponds to a pose transformation matrix. The pose transformation matrix corresponding to any set of local point cloud data includes transforming the query key frame into any one through any set of local point cloud data. At least one of a rotation transformation parameter and a translation transformation parameter of the matching frame.

For example, for the multiple sets of local point cloud data corresponding to the sub-image of the query key frame and the global point cloud data of the sub-image of any matching frame, a registration algorithm is used to perform point cloud registration. In one possible implementation, multiple sets of local point cloud data corresponding to the sub-image of the query key frame and global point cloud data of the sub-image of any matching frame are used during registration, instead of the sub-image of any matching frame. Local point cloud data can avoid the registration results being unreliable due to the small overlap area between the multiple sets of local point cloud data corresponding to the sub-image of the query key frame and the local point cloud data of the sub-image of any matching frame. The phenomenon. The embodiments of this application do not limit the registration method. For example, the registration method can be: point-to-surface ICP (IterativeClosest Point, iterative closest point), point-to-point ICP, GICP (Generalized Iterative Closest Point, comprehensive iterative closest point), etc. Traditional methods and deep learning methods such as D3Feat (Dense Detection and Description Feat, dense detection and description function), FCGF (Fully Convolutional Geometric Features, full convolutional geometric features), PREDATOR (registration method of low-overlap three-dimensional point cloud). Each set of local point cloud data corresponding to the sub-image of the query key frame and the global point cloud data of any matching frame sub-image are subjected to the registration calculation of any of the above registration algorithms to obtain a corresponding pose transformation matrix. The pose transformation matrix corresponding to the set of local point cloud data includes at least one of a rotation transformation parameter and a translation transformation parameter that transforms the query key frame into any matching frame through any set of local point cloud data.

Step 2044: Calculate the mean value of the pose transformation matrix that meets the requirements among the multiple pose transformation matrices, and use the calculated result as the registration result of the query key frame and any matching frame.

According to each set of local point cloud data corresponding to the sub-image of the query key frame and the global point cloud data of any matching frame sub-image, a corresponding pose transformation matrix is obtained through the registration calculation of any of the above registration algorithms. , calculate the first average transformation matrix of all pose transformation matrices of the query key frame and the matching frame, and calculate the difference between each pose transformation matrix and the first average transformation matrix respectively. Compare the difference between each pose transformation matrix and the first average transformation matrix with the transformation matrix threshold, filter out the pose transformation matrix whose difference with the first average transformation matrix is less than the transformation matrix threshold, and transform the pose according to the filtered out The matrix recalculates the mean to obtain the registration result of the query key frame and any matching frame.

For example, a registration result can be obtained by calculation between the query key frame and any matching frame of the query key frame, and the number of matching frames of the query key frame is the number of registration results finally obtained for the query key frame.

Among them, for special situations that occur, for example, when only the overlapping area ratio of two sets of local point cloud data in the local point cloud data corresponding to the sub-image of the query key frame and the global point cloud data of the sub-image of the matching frame meets the proportion requirements, and The differences between the two obtained pose transformation matrices and the first average transformation matrix both exceed the transformation matrix threshold. At this time, the pose transformation matrix corresponding to the local point cloud data with the largest overlap area is selected as the registration result.

In a possible implementation, after determining the registration results of the query key frame and each matching frame of the query key frame, for any matching frame, according to the registration result of the query key frame and any matching frame and the registration The transformation matrix calculates the pose error of the query key frame, and the registration transformation matrix includes at least one of a rotation transformation parameter and a translation transformation parameter that transforms the query key frame into any matching frame through the global point cloud data of the query key frame.

In one possible implementation, the inverse matrix of the registration transformation matrix is multiplied by any registration result of the query key frame to obtain the pose error of the matching frame corresponding to the query key frame and the registration result.

Illustratively, based on the above registration result T _f of the matching frame p of the query key frame q, the pose error T _diff of the query key frame q and the matching frame p is calculated according to T _diff = (T _pq ) ^-1 T _f , including Rotation error and translation error.

After obtaining the pose error between the query key frame and each matching frame of the query key frame, in response to the pose error being greater than the pose error threshold, it is determined that the registration result of any matching frame and the query key frame meets the requirements.

Compare each pose error of the query key frame with the pose error threshold, and filter out the query key frames whose maximum pose error exceeds the pose error threshold. The pose error threshold refers to a value set in advance according to the required map accuracy, including rotation threshold and translation threshold. For example, the rotation threshold is the angle threshold. For the rotation error within the obtained pose error, first convert the rotation matrix into Euler angles. If the error in one of the three directions of Euler angles is greater than the rotation threshold, the pose error is considered to exceed the pose error threshold; For the translation error within the obtained pose error, if the error in one of the three directions xyz is greater than the translation threshold, the pose error is considered to exceed the pose error threshold. In a possible implementation, if at least one of the rotation error and translation error within the obtained pose error is greater than its corresponding rotation threshold or translation threshold, it is considered that the calculated registration result of the pose error satisfies Require.

Step 205: In response to the existence of a registration result that meets the requirements, determine the target grid as a grid with ghosting.

In a possible implementation, after applying the method provided by the embodiment of the present application to determine the grids with ghosts, the grids with ghosts can be merged, and the ghost areas can be determined based on the merged grids.

For example, after it is determined that the registration result of any matching frame and the query key frame meets the requirements, the target grid, that is, the grid where the query key frame is located, is determined as a grid with ghosting. After determining all the grids with ghosts in the point cloud map, merge the adjacent ghost grids in the point cloud map to obtain the areas with ghosts in the point cloud map.

Taking the schematic diagram of the point cloud map after the grid where ghosting is determined as shown in Figure 10 as an example, the grid where cross symbols exist is the grid where ghosting is determined to exist. Merge adjacent grids with ghosts in the point cloud map to determine multiple regions with ghosts, corresponding to the areas selected by the thick solid lines in the figure.

Optionally, the method provided by the embodiment of the present application also includes: repairing the point cloud trajectory included in the ghost area. For example, after determining the area where ghosting exists in the point cloud map, the corresponding radar scanning area is determined based on the ghosting area. In this area, the radar re-scans multiple times to repair the point cloud trajectory corresponding to the area, remove ghosts in the point cloud trajectory, and ensure the quality of the point cloud map.

In summary, the method provided by the embodiments of the present application divides the point cloud map into grids and converts the large-scale point cloud map into point cloud trajectories in multiple small-scale grids with finer granularity, thereby achieving point cloud mapping. Full coverage of ghost detection in cloud maps. Select the query key frame and the matching frame, synthesize the query key frame sub-image and the matching frame sub-image, and conduct multiple rounds of registration calculations between at least one local point cloud data of the query key frame sub-image and the matching frame sub-image to complete the point cloud map Ghost detection and multiple registrations improve the accuracy of detection, so that an accurate high-precision map can be constructed based on the point cloud map after ghost detection, and then the normal driving of autonomous vehicles can be ensured based on the accurate high-precision map. Ghost detection is performed through the registration algorithm. Since there is no need to extract specific elements, the scene restrictions on ghost detection are reduced.

Referring to Figure 11, an embodiment of the present application provides a ghost detection device for point cloud maps. The device includes:

The dividing module 1101 is used to divide the point cloud map to be detected into multiple grids. The point cloud map includes multi-frame point cloud trajectories, and each grid includes local point cloud trajectories in the multi-frame point cloud trajectories;

Determining module 1102, used to determine the query key frame from the local point cloud trajectory included in the target grid;

The determination module 1102 is also used to determine at least one matching frame of the query key frame in the multi-frame point cloud trajectory, where the matching frame is the local point cloud trajectory within the reference range where the query key frame is located;

The determination module 1102 is also used to determine the registration results of the query key frame and each matching frame of the query key frame;

The determination module 1102 is also configured to determine the target grid as a grid with ghosting in response to the existence of a registration result that meets the requirements.

In a possible implementation, the determination module 1102 is used to determine the number of road layers in the target grid; determine a reference number of point cloud trajectories as query key frames from the local point cloud trajectories included in the target grid. The reference number Determined based on the number of road layers.

In a possible implementation, the device further includes: a filtering module for filtering out local point cloud trajectories within the reference range from multi-frame point cloud trajectories;

The determination module 1102 is configured to determine, in response to the reference range including multiple local point cloud trajectories, the local point cloud trajectories that satisfy the time interval among the multiple local point cloud trajectories as matching frames of the query key frame.

In a possible implementation, the determination module 1102 is used to determine the subgraph of the query key frame. The subgraph of the query key frame is obtained based on the point cloud trajectory included in the query key frame; for the query key frame and any matching frame, Determine the subgraph of any matching frame, which is obtained based on the point cloud trajectory included in any matching frame; determine multiple pose transformation matrices based on the subgraph of the query key frame and the subgraph of any matching frame , any pose transformation matrix includes at least one of a rotation transformation parameter and a translation transformation parameter that transforms the query key frame into any matching frame;

The device also includes: a calculation module for averaging the pose transformation matrices that meet the requirements among the multiple pose transformation matrices, and using the calculated result as a registration result between the query key frame and any matching frame.

In a possible implementation, the merging module is used to merge the point cloud trajectories included in the query key frame and the relevant point cloud trajectories of the query key frame, to obtain a subgraph of the query key frame, and the relevant point cloud of the query key frame. The trajectories include point cloud trajectories within the target mesh adjacent to the query keyframe timestamp.

In a possible implementation, the merging module is used to merge the point cloud trajectory included in any matching frame with the relevant point cloud trajectory of any matching frame to obtain a sub-image of any matching frame. The relevant point cloud trajectories include point cloud trajectories adjacent to the timestamp of any matching frame within the grid where any matching frame is located.

In a possible implementation, the determination module 1102 is used to determine multiple sets of local point cloud data corresponding to the sub-image of the query key frame and global point cloud data corresponding to the sub-image of any matching frame;

The device also includes: a registration module, used to register multiple sets of local point cloud data with global point cloud data to obtain multiple pose transformation matrices. A set of local point cloud data corresponds to a pose transformation matrix. Any one The pose transformation matrix corresponding to the set of local point cloud data includes at least one of a rotation transformation parameter and a translation transformation parameter that transforms the query key frame into any matching frame through any set of local point cloud data.

In a possible implementation, the filtering module is used to filter out the point cloud data that meets the proportion requirements according to different regions from the global point cloud data included in the sub-graph of the query key frame, and filter out the point cloud data that meets the proportion requirements in each region. The required point cloud data is used as a set of local point cloud data corresponding to the subgraph of the query key frame.

In a possible implementation, the calculation module is also used to calculate, for any matching frame, the pose error of the query key frame based on the registration result and the registration transformation matrix between the query key frame and any matching frame. The transformation matrix includes at least one of a rotation transformation parameter and a translation transformation parameter that transforms the query key frame into any matching frame through the global point cloud data of the query key frame;

The determination module 1102 is also configured to determine that the registration result of any matching frame and the query key frame meets the requirements in response to the pose error being greater than the pose error threshold.

In a possible implementation, the determination module 1102 is also used to merge grids with ghosts, and determine the ghost area based on the merged grids;

The device also includes: a repair module for repairing the point cloud trajectory included in the ghost area.

The device provided by the embodiment of the present application divides the point cloud map into grids and converts the large-scale point cloud map into point cloud trajectories in multiple small-scale grids with finer granularity, thereby realizing the point cloud map in the point cloud map. Full coverage of ghost detection. Select the query key frame and the matching frame, synthesize the query key frame sub-image and the matching frame sub-image, and conduct multiple rounds of registration calculations between at least one local point cloud data of the query key frame sub-image and the matching frame sub-image to complete the point cloud map Ghost detection and multiple registrations improve the accuracy of detection, so that an accurate high-precision map can be constructed based on the point cloud map after ghost detection, and then the normal driving of autonomous vehicles can be ensured based on the accurate high-precision map. Ghost detection is performed through the registration algorithm. Since there is no need to extract specific elements, the scene restrictions on ghost detection are reduced.

It should be noted that when the device provided in the above embodiment implements its functions, only the division of the above functional modules is used as an example. In actual application, the above functions can be allocated to different functional modules according to needs, that is, the equipment The internal structure is divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be described again here.

It should be noted that the terms "first", "second", etc. (if present) in the description and claims of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It is to be understood that the data so used are interchangeable under appropriate circumstances so that the embodiments of the application described herein can be practiced in sequences other than those illustrated or described herein. The implementations described in the following exemplary embodiments do not represent all implementations consistent with this application. Rather, they are merely examples of apparatus and methods consistent with aspects of the application as detailed in the appended claims.

Figure 12 is a schematic structural diagram of a server provided by an embodiment of the present application. The server may vary greatly due to different configurations or performance, and may include one or more processors 1201 and one or more memories 1202. The processor 1201 is, for example, a CPU (Central Processing Units, central processing unit). Among them, at least one computer program is stored in the one or more memories 1202, and the at least one computer program is loaded and executed by the one or more processors 1201, so that the server implements the point cloud map provided by the above method embodiments. Ghost detection method. Of course, the server can also have components such as wired or wireless network interfaces, keyboards, and input and output interfaces to facilitate input and output. The server can also include other components for implementing device functions, which will not be described again here.

Figure 13 is a schematic structural diagram of a ghost detection device for point cloud maps provided by an embodiment of the present application. The device may be a terminal, for example, a smartphone, a tablet, a laptop or a desktop computer. The terminal may also be called user equipment, portable terminal, laptop terminal, desktop terminal, and other names.

Generally, the terminal includes: a processor 1301 and a memory 1302.

The processor 1301 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 1301 can adopt at least one hardware form among DSP (Digital Signal Processing, digital signal processing), FPGA (Field-Programmable Gate Array, field programmable gate array), and PLA (Programmable Logic Array, programmable logic array). accomplish. The processor 1301 may also include a main processor and a co-processor. The main processor is a processor used to process data in the wake-up state, also called CPU (Central Processing Unit, central processing unit); the co-processor is A low-power processor used to process data in standby mode. In some embodiments, the processor 1301 may be integrated with a GPU (Graphics Processing Unit, image processor), and the GPU is responsible for rendering and drawing content to be displayed on the display screen. In some embodiments, the processor 1301 may also include an AI (Artificial Intelligence, artificial intelligence) processor, which is used to process computing operations related to machine learning.

Memory 1302 may include one or more computer-readable storage media, which may be non-transitory. Memory 1302 may also include high-speed random access memory, and non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 1302 is used to store at least one instruction, and the at least one instruction is used to be executed by the processor 1301 to enable the terminal to implement the method embodiments of the present application. Provided ghost detection method.

In some embodiments, the terminal optionally further includes: a peripheral device interface 1303 and at least one peripheral device. The processor 1301, the memory 1302 and the peripheral device interface 1303 may be connected through a bus or a signal line. Each peripheral device can be connected to the peripheral device interface 1303 through a bus, a signal line, or a circuit board. Specifically, the peripheral device includes: at least one of a radio frequency circuit 1304, a display screen 1305, a camera component 1306, an audio circuit 1307, a positioning component 1308 and a power supply 1309.

The peripheral device interface 1303 may be used to connect at least one I/O (Input/Output) related peripheral device to the processor 1301 and the memory 1302 . In some embodiments, the processor 1301, the memory 1302, and the peripheral device interface 1303 are integrated on the same chip or circuit board; in some other embodiments, any one of the processor 1301, the memory 1302, and the peripheral device interface 1303 or Both of them can be implemented on separate chips or circuit boards, which is not limited in this embodiment.

The radio frequency circuit 1304 is used to receive and transmit RF (Radio Frequency, radio frequency) signals, also called electromagnetic signals. Radio frequency circuit 1304 communicates with communication networks and other communication devices through electromagnetic signals. The radio frequency circuit 1304 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals into electrical signals. Optionally, the radio frequency circuit 1304 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, and the like. Radio frequency circuitry 1304 can communicate with other terminals through at least one wireless communication protocol. The wireless communication protocol includes but is not limited to: metropolitan area network, various generations of mobile communication networks (2G, 3G, 4G and 5G), wireless local area network and/or WiFi (Wireless Fidelity, wireless fidelity) network. In some embodiments, the radio frequency circuit 1304 may also include NFC (Near Field Communication) related circuits, which is not limited in this application.

The display screen 1305 is used to display UI (User Interface, user interface). The UI can include graphics, text, icons, videos, and any combination thereof. When display screen 1305 is a touch display screen, display screen 1305 also has the ability to collect touch signals on or above the surface of display screen 1305 . The touch signal can be input to the processor 1301 as a control signal for processing. At this time, the display screen 1305 can also be used to provide virtual buttons and/or virtual keyboards, also called soft buttons and/or soft keyboards. In some embodiments, there may be one display screen 1305, which is disposed on the front panel of the terminal; in other embodiments, there may be at least two display screens 1305, which are respectively disposed on different surfaces of the terminal or in a folding design; in another In some embodiments, the display screen 1305 may be a flexible display screen disposed on a curved surface or a folding surface of the terminal. Even, the display screen 1305 can also be set in a non-rectangular irregular shape, that is, a special-shaped screen. The display screen 1305 can be made of LCD (Liquid Crystal Display, liquid crystal display), OLED (Organic Light-Emitting Diode, organic light-emitting diode) and other materials.

The camera component 1306 is used to capture images or videos. Optionally, the camera assembly 1306 includes a front camera and a rear camera. Usually, the front camera is set on the front panel of the terminal, and the rear camera is set on the back of the terminal. In some embodiments, there are at least two rear cameras, one of which is a main camera, a depth-of-field camera, a wide-angle camera, and a telephoto camera, so as to realize the integration of the main camera and the depth-of-field camera to realize the background blur function. Integrated with a wide-angle camera to achieve panoramic shooting and VR (Virtual Reality, virtual reality) shooting functions or other fusion shooting functions. In some embodiments, camera assembly 1306 may also include a flash. The flash can be a single color temperature flash or a dual color temperature flash. Dual color temperature flash refers to a combination of warm light flash and cold light flash, which can be used for light compensation under different color temperatures.

Audio circuitry 1307 may include a microphone and speakers. The microphone is used to collect sound waves from the user and the environment, and convert the sound waves into electrical signals that are input to the processor 1301 for processing, or to the radio frequency circuit 1304 to implement voice communication. For the purpose of stereo collection or noise reduction, there can be multiple microphones, which are respectively installed at different parts of the terminal. The microphone can also be an array microphone or an omnidirectional collection microphone. The speaker is used to convert electrical signals from the processor 1301 or the radio frequency circuit 1304 into sound waves. The loudspeaker can be a traditional membrane loudspeaker or a piezoelectric ceramic loudspeaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert electrical signals into sound waves that are audible to humans, but also convert electrical signals into sound waves that are inaudible to humans for purposes such as ranging. In some embodiments, audio circuitry 1307 may also include a headphone jack.

The positioning component 1308 is used to locate the current geographical location of the terminal to implement navigation or LBS (Location Based Service). The positioning component 1308 may be a positioning component based on the American GPS (Global Positioning System, Global Positioning System), China's Beidou system, Russia's Galileo system, or the European Union's Galileo system.

The power supply 1309 is used to power various components in the terminal. Power source 1309 may be AC, DC, disposable batteries, or rechargeable batteries. When the power source 1309 includes a rechargeable battery, the rechargeable battery may support wired charging or wireless charging. The rechargeable battery can also be used to support fast charging technology.

In some embodiments, the terminal also includes one or more sensors 1310. The one or more sensors 1310 include, but are not limited to: an acceleration sensor 1311, a gyroscope sensor 1312, a pressure sensor 1313, a fingerprint sensor 1314, an optical sensor 1315, and a proximity sensor 1316.

The acceleration sensor 1311 can detect the acceleration on the three coordinate axes of the coordinate system established by the terminal. For example, the acceleration sensor 1311 can be used to detect the components of gravity acceleration on three coordinate axes. The processor 1301 can control the display screen 1305 to display the user interface in a horizontal view or a vertical view according to the gravity acceleration signal collected by the acceleration sensor 1311. The acceleration sensor 1311 can also be used to collect game or user motion data.

The gyro sensor 1312 can detect the body direction and rotation angle of the terminal, and the gyro sensor 1312 can cooperate with the acceleration sensor 1311 to collect the user's 3D movements on the terminal. Based on the data collected by the gyro sensor 1312, the processor 1301 can implement the following functions: motion sensing (such as changing the UI according to the user's tilt operation), image stabilization during shooting, game control, and inertial navigation.

The pressure sensor 1313 may be provided on the side frame of the terminal and/or on the lower layer of the display screen 1305 . When the pressure sensor 1313 is disposed on the side frame of the terminal, it can detect the user's holding signal of the terminal, and the processor 1301 performs left and right hand identification or quick operation based on the holding signal collected by the pressure sensor 1313. When the pressure sensor 1313 is provided on the lower layer of the display screen 1305, the processor 1301 controls the operability controls on the UI interface according to the user's pressure operation on the display screen 1305. The operability control includes at least one of a button control, a scroll bar control, an icon control, and a menu control.

The fingerprint sensor 1314 is used to collect the user's fingerprint. The processor 1301 identifies the user's identity based on the fingerprint collected by the fingerprint sensor 1314, or the fingerprint sensor 1314 identifies the user's identity based on the collected fingerprint. When the user's identity is recognized as a trusted identity, the processor 1301 authorizes the user to perform relevant sensitive operations. The sensitive operations include unlocking the screen, viewing encrypted information, downloading software, making payments, and changing settings. The fingerprint sensor 1314 may be provided on the front, back or side of the terminal. When a physical button or manufacturer's logo (trademark) is provided on the terminal, the fingerprint sensor 1314 can be integrated with the physical button or manufacturer's logo.

The optical sensor 1315 is used to collect ambient light intensity. In one embodiment, the processor 1301 can control the display brightness of the display screen 1305 according to the ambient light intensity collected by the optical sensor 1315. Specifically, when the ambient light intensity is high, the display brightness of the display screen 1305 is increased; when the ambient light intensity is low, the display brightness of the display screen 1305 is decreased. In another embodiment, the processor 1301 can also dynamically adjust the shooting parameters of the camera assembly 1306 according to the ambient light intensity collected by the optical sensor 1315.

The proximity sensor 1316, also called a distance sensor, is usually provided on the front panel of the terminal. The proximity sensor 1316 is used to collect the distance between the user and the front of the terminal. In one embodiment, when the proximity sensor 1316 detects that the distance between the user and the front of the terminal gradually becomes smaller, the processor 1301 controls the display screen 1305 to switch from the bright screen state to the closed screen state; when the proximity sensor 1316 detects that the user When the distance from the front of the terminal gradually increases, the processor 1301 controls the display screen 1305 to switch from the screen-off state to the screen-on state.

Those skilled in the art can understand that the structure shown in Figure 13 does not constitute a limitation of the terminal, and may include more or fewer components than shown, or combine certain components, or adopt different component arrangements.

In an exemplary embodiment, a computer device is also provided. The computer device includes a processor and a memory, and at least one computer program is stored in the memory. The at least one computer program is loaded and executed by one or more processors, so that the computer device implements any of the above ghost detection methods.

In an exemplary embodiment, a computer-readable storage medium is also provided. At least one computer program is stored in the computer-readable storage medium. The at least one computer program is loaded and executed by a processor of the computer device, so that the computer Methods to implement any of the above ghost detection methods.

In a possible implementation, the computer-readable storage medium may be a read-only memory (Read-OnlyMemory, ROM), a random access memory (Random Access Memory, RAM), a read-only compact disc (Compact DiscRead-Only Memory, CD). -ROM), tapes, floppy disks and optical data storage devices, etc.

In an exemplary embodiment, a computer program product or computer program is also provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs any of the above ghost detection methods.

It should be noted that the information (including but not limited to user equipment information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.) and signals involved in this application, All are authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions. For example, the grids, query key frames, and matching frames involved in this application were all obtained with full authorization.

It should be understood that "plurality" mentioned in this article means two or more. "And/or" describes the relationship between associated objects, indicating that there can be three relationships. For example, A and/or B can mean: A exists alone, A and B exist simultaneously, and B exists alone. The character "/" generally indicates that the related objects are in an "or" relationship.

The above are only exemplary embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the principles of the present application shall be included in the protection scope of the present application. Inside.

Claims (10)

1. A ghost detection method for point cloud maps, characterized in that the method includes: Divide the point cloud map to be detected into multiple grids, the point cloud map includes a multi-frame point cloud trajectory, and each grid includes a local point cloud trajectory in the multi-frame point cloud trajectory; Determine query keyframes from local point cloud trajectories included in the target mesh; Determine at least one matching frame of the query key frame in the multi-frame point cloud trajectory, where the matching frame is a local point cloud trajectory within the reference range where the query key frame is located; Determine the registration results of the query key frame and each matching frame of the query key frame; In response to the presence of a registration result that satisfies the requirements, the target grid is determined to be a grid with ghosts present. 2. The method according to claim 1, characterized in that determining the query key frame from the local point cloud trajectory included in the target grid includes: Determine the number of road layers in the target grid; A reference number of point cloud trajectories are determined as query key frames from the local point cloud trajectories included in the target grid, and the reference number is determined based on the number of road layers. 3. The method of claim 1, wherein determining at least one matching frame of the query key frame in the multi-frame point cloud trajectory includes: Filter out local point cloud trajectories within the reference range from the multi-frame point cloud trajectories; In response to the reference range including multiple local point cloud trajectories, the local point cloud trajectories that satisfy the time interval among the multiple local point cloud trajectories are determined as matching frames of the query key frame. 4. The method of claim 1, wherein determining the registration results of the query key frame and each matching frame of the query key frame includes: Determine the subgraph of the query key frame, which is obtained based on the point cloud trajectory included in the query key frame; For any matching frame of the query key frame, determine a sub-image of the any matching frame, and the sub-image of any matching frame is obtained based on the point cloud trajectory included in the any matching frame; A plurality of pose transformation matrices are determined based on the sub-image of the query key frame and the sub-image of any matching frame, and any pose transformation matrix includes rotation transformation parameters that transform the query key frame into any matching frame. and at least one of translation transformation parameters; The pose transformation matrix that meets the requirements among the multiple pose transformation matrices is averaged, and the calculated result is used as the registration result of the query key frame and any matching frame. 5. The method of claim 4, wherein determining the subgraph of the query key frame includes: Merge the point cloud trajectories included in the query key frame and the relevant point cloud trajectories of the query key frame to obtain a subgraph of the query key frame. The relevant point cloud trajectories of the query key frame include the target network The point cloud trajectory adjacent to the query keyframe timestamp in the grid. 6. The method according to claim 4, wherein determining the sub-image of any matching frame includes: Merge the point cloud trajectory included in any matching frame with the relevant point cloud trajectory of any matching frame to obtain a sub-image of any matching frame, and the relevant point cloud trajectory of any matching frame includes The point cloud trajectory adjacent to the timestamp of any matching frame within the grid where any matching frame is located. 7. The method of claim 4, wherein determining a plurality of pose transformation matrices based on the subgraph of the query key frame and the subgraph of any matching frame includes: Determine multiple sets of local point cloud data corresponding to the sub-image of the query key frame and global point cloud data corresponding to the sub-image of any matching frame; The multiple sets of local point cloud data are registered with the global point cloud data respectively to obtain multiple pose transformation matrices. One set of local point cloud data corresponds to one pose transformation matrix, and any set of local point cloud data corresponds to The pose transformation matrix includes at least one of a rotation transformation parameter and a translation transformation parameter that transforms the query key frame into any matching frame through the any set of local point cloud data. 8. The method according to claim 7, characterized in that determining multiple sets of local point cloud data corresponding to the sub-image of the query key frame includes: In the global point cloud data included in the sub-picture of the query key frame, point cloud data that meets the proportion requirements are filtered out according to different areas, and the point cloud data that meets the proportion requirements filtered out in each area are used as the query key frame. A set of local point cloud data corresponding to the subgraph. 9. The method according to claim 1, characterized in that, after determining the registration results of the query key frame and each matching frame of the query key frame, it further includes: For any matching frame, the pose error of the query key frame is calculated according to the registration result and the registration transformation matrix of the query key frame and any matching frame, and the registration transformation matrix includes the query key frame and the registration transformation matrix. The global point cloud data of the key frame transforms the query key frame into at least one of the rotation transformation parameter and the translation transformation parameter of any matching frame; In response to the pose error being greater than the pose error threshold, it is determined that the registration result of any matching frame and the query key frame meets the requirements. 10. The method according to any one of claims 1 to 9, characterized in that after determining the target grid as a grid with ghosts, it further includes: Merge the meshes with ghosts and determine the ghosting area based on the merged meshes; Repair the point cloud trajectory included in the ghost area.