WO2022141220A1

WO2022141220A1 - Point cloud processing method and device, ranging device, and movable platform

Info

Publication number: WO2022141220A1
Application number: PCT/CN2020/141497
Authority: WO
Inventors: 陈涵; 刘政; 李延召
Original assignee: 深圳市大疆创新科技有限公司
Priority date: 2020-12-30
Filing date: 2020-12-30
Publication date: 2022-07-07

Abstract

A point cloud processing method, comprising: obtaining a plurality of point cloud points collected within a current frame duration (202); determining an object of interest among the plurality of point cloud points (204); and obtaining a target point cloud point associated with the object of interest from other point cloud frames other than a current frame, and adding the target point cloud point to the plurality of point cloud points to obtain a target point cloud frame (206). The method can solve the technical problem of the low point cloud density or the small number of point cloud points of the object of interest in the generated point cloud frame.

Description

Point cloud processing method and device, ranging device and movable platform

technical field

The present application relates to the technical field of point cloud processing, and in particular, to a point cloud processing method, a point cloud processing device, a ranging device, a movable platform, and a computer-readable storage medium.

Background technique

Lidar can scan the scene, so as to obtain multiple point cloud points corresponding to the scene. However, according to its own scanning method, the LiDAR scans the point cloud frame with a low number of point cloud points or point cloud density of the object of interest, resulting in inaccurate identification of the object of interest based on the point cloud frame, and the direct output display Nor does it provide enough information associated with the object of interest.

SUMMARY OF THE INVENTION

In view of this, the embodiments of the present application provide a point cloud processing method, a point cloud processing device, a ranging device, a movable platform and a computer-readable storage medium, one of the purposes is to solve the object of interest in the generated point cloud frame The technical problem of the low density of the point cloud or the low number of point cloud points.

A first aspect of the embodiments of the present application provides a point cloud processing method, including:

Obtain multiple point cloud points collected within one frame;

determining an object of interest in the plurality of point cloud points;

The target point cloud point associated with the object of interest is obtained from other point cloud frames other than the current frame, and added to the plurality of point cloud points to obtain the target point cloud frame.

A second aspect of an embodiment of the present application provides a point cloud processing device, including: a processor and a memory storing a computer program, where the processor implements the following steps when executing the computer program:

Obtain multiple point cloud points collected within the current frame duration;

determining an object of interest in the plurality of point cloud points;

A target point cloud point associated with the object of interest is acquired from other point cloud frames other than the current frame, and added to the plurality of point cloud points to obtain a target point cloud frame.

A third aspect of the embodiments of the present application provides a distance measuring device, including:

an emitter for emitting light pulses;

a detector for detecting the light beam reflected by the light pulse;

A processor and a memory in which a computer program is stored, the processor implementing the following steps when executing the computer program:

Obtain multiple point cloud points collected within the current frame duration;

determining an object of interest in the plurality of point cloud points;

A fourth aspect of the embodiments of the present application provides a movable platform, including:

body;

a drive device connected to the body;

A ranging device mounted on the body, the ranging device is used to implement the following steps:

Obtain multiple point cloud points collected within the current frame duration;

determining an object of interest in the plurality of point cloud points;

A fifth aspect of the embodiments of the present application provides a computer-readable storage medium, where the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, implements the point cloud processing method provided by the embodiments of the present application.

In the point cloud processing method provided in the embodiment of the present application, the target point cloud points associated with the object of interest can be obtained from other point cloud frames other than the current frame, and added to multiple point cloud points collected within the current frame duration, thereby obtaining In the target point cloud frame, there can be more point cloud points corresponding to the object of interest, and the density of the point cloud corresponding to the object of interest can also be higher. By outputting and displaying the target point cloud frame, the user can obtain more information about the object of interest, and processing the target point cloud frame algorithmically can also improve the processing effect of the algorithm, such as improving the recognition accuracy of the object of interest. .

Description of drawings

In order to illustrate the technical solutions in the embodiments of the present application more clearly, the following briefly introduces the drawings that are used in the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can also be obtained from these drawings without creative labor.

FIG. 1 is a schematic diagram of point cloud frame generation provided by an embodiment of the present application.

FIG. 2 is a flowchart of a point cloud processing method provided by an embodiment of the present application.

FIG. 3 is a scene diagram 1 in which a pedestrian and a ranging device have relative motion according to an embodiment of the present application.

FIG. 4 is a second scene diagram in which a pedestrian and a ranging device have relative motion according to an embodiment of the present application.

FIG. 5 is a schematic structural diagram of a point cloud processing apparatus provided by an embodiment of the present application.

FIG. 6 is a schematic structural diagram of a ranging apparatus provided by an embodiment of the present application.

FIG. 7 is a schematic structural diagram of a movable platform provided by an embodiment of the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

The distance measuring device can measure the distance between the target and the device. In one embodiment, the ranging device can measure the time-of-flight (TOF, Time-of-Flight) to achieve ranging. Specifically, the ranging device can include a transmitter and a receiver. The transmitter can transmit a sequence of light pulses, and the beam can be reflected after reaching the surface of the target. The time of flight of light calculates the distance between the device and the surface of the target.

In one embodiment, the ranging device may be a lidar. The lidar can collect multiple point cloud points corresponding to the scene through the above-mentioned principle of measuring flight time, and can generate a frame of point cloud frame by using the point cloud points collected within a fixed time (one frame duration). Referring to FIG. 1 , FIG. 1 is a schematic diagram of point cloud frame generation provided by an embodiment of the present application. As shown in FIG. 1 , the fixed time (one frame duration) may be 0.1S, that is, the point cloud points contained in one frame of point cloud frame may be collected within 0.1S duration. In an example, one frame of point cloud frame can be generated every 0.1S, for example, the point cloud points collected from 0-0.1S can be used to generate the first frame, and the point cloud points collected from 0.1-0.2S can be used to generate Second frame. In an example, point cloud points can be reused between adjacent point cloud frames. For example, the point cloud points collected from 0-0.1S can be used to generate the first frame, and the point cloud points collected from 0.05 to 0.15S can be used to generate the first frame. For generating the second frame, the point cloud points collected at 0.05-0.1S are multiplexed by the first frame and the second frame.

It can be understood that, in one embodiment, the generated point cloud frame can be used for output display. In one embodiment, the generated point cloud frame can also be used by an algorithm to perform related point cloud processing, such as object detection, semantic segmentation, target tracking, and the like.

Since people pay more attention to the object of interest in the scene, people hope that the point cloud point of the object of interest in the point cloud frame can be more, or the point cloud density of the object of interest can be higher. However, when the lidar collects point cloud points according to its own scanning method, it does not collect more objects of interest, so in the obtained point cloud frame, the number of point cloud points or point cloud density of the object of interest cannot meet the requirements. Require.

Based on the above problem, an embodiment of the present application provides a point cloud processing method. Referring to FIG. 2 , FIG. 2 is a flowchart of a point cloud processing method provided by an embodiment of the present application. The method may include the following steps:

S202: Acquire multiple point cloud points collected within the duration of the current frame.

S204. Determine the object of interest in the multiple point cloud points.

S206. Obtain a target point cloud point associated with the object of interest from other point cloud frames other than the current frame, and add the multiple point cloud points to obtain a target point cloud frame.

In one embodiment, multiple point cloud points collected during the current frame duration can be stored in a defined buffer, and the target point cloud points obtained from the other point cloud frames can also be stored in the defined buffer. The buffer is used to generate one frame of the target point cloud frame by using the point cloud points stored in the buffer.

In one embodiment, the object of interest may be an object of interest, and the target point cloud point may be a point cloud point corresponding to the object of interest. The objects of interest can be various contents in the scene. In the scene of automatic driving, the objects of interest can be people, the sky, cars, trees, etc., and the target point cloud points can be the point cloud points corresponding to these contents in the scene. .

In one embodiment, the object of interest may be user-specified. For example, the user can specify that a person is an object of interest, and then a target point cloud point corresponding to the person can be obtained from the other point cloud frames and added to the plurality of point cloud points. For the target point cloud points in other point cloud frames, in an implementation manner, it can be determined according to the semantic information corresponding to the other point cloud frames. Specifically, semantic segmentation may be performed on the other point cloud frames, and the category corresponding to each point cloud point in the other point cloud frames may be determined, so that the category may be the point cloud point of the target category corresponding to the object of interest Determined as the target point cloud point.

The categories corresponding to the above point cloud points may include: motion state category, attribute category, structure category, size category, and the like. Among them, the motion state category may include motion state, static state, motion direction (front, back, left, right, up, down, etc.), motion speed (speed, slow, etc.), etc., and attribute categories may include: people, animals, vehicles, buildings, plants, roadblocks, ground, The sky, etc., the structure category can include the distribution of point clouds, such as uniform distribution of point clouds, uneven distribution of point clouds, and the size category can include sizes of various sizes, such as large size, medium size, and small size. It can be understood that the above-mentioned descriptions of expressing degrees such as fast and slow, large, medium and small can be distinguished by setting corresponding thresholds during specific implementation.

When determining the object of interest in the multiple point cloud points, in one embodiment, the category corresponding to each point cloud point in the multiple point cloud points may be determined, according to the target corresponding to the object of interest The category may determine the region corresponding to the object of interest in the plurality of point cloud points. Here, when the category corresponding to each point cloud point is determined, in one embodiment, the determination may be performed by performing semantic segmentation on the plurality of point cloud points.

When the object of interest and the ranging device remain relatively stationary, the position of the object of interest in other point cloud frames is the same or similar to the position of the object of interest in the multiple point cloud points collected during the current frame, then the The target point cloud points corresponding to the object of interest in other point cloud frames are directly added to the plurality of point cloud points, and in the obtained target point cloud frame, the point cloud points corresponding to the object of interest can be more concentrated, and the Objects of interest are sharper on the frame. However, when the object of interest and the ranging device have relatively large relative motions, the position of the object of interest in other point cloud frames is different from the position of the object of interest in the multiple point cloud points, as shown in Figure 3 In the example shown, the object of interest may be a pedestrian, the position of the pedestrian in the plurality of point cloud points is the B position, and the position of the pedestrian in the other point cloud frames is the A position. In this way, in one embodiment, the target point cloud points corresponding to the object of interest in other point cloud frames can be directly added to the plurality of point cloud points, and the ghost effect can be obtained in the target point cloud frame. object of interest. In an embodiment, the position of the target point cloud point may be corrected, and the corrected target point cloud point may be added to the plurality of point cloud points, and in the obtained target point cloud frame, the corresponding object of interest Point cloud points can be concentrated in the same area, and objects of interest can be more clearly defined.

When performing position correction on the target point cloud point, in one embodiment, the position of the object of interest in the plurality of point cloud points may be compared with the object of interest in the other point cloud frames. The position of the target point cloud point is corrected. In one example, semantic segmentation may be performed first on the multiple point cloud points and the other point cloud frames, and according to the semantic information obtained by the segmentation, it is determined that the object of interest is located in the multiple point cloud points and the other point clouds. The position in the frame, in the example of Figure 3, the position of the object of interest pedestrian in the multiple point cloud points is the B position, and the position of the object of interest in the other point cloud frames is the A position, then according to The position difference between the A position and the B position can be corrected for the target point cloud point corresponding to the object of interest in the other point cloud frames, that is, the entire target point cloud point can be corrected from the A position to the B position. The target point cloud points after position correction can be added to the multiple point cloud points, so that a target point cloud frame with a clear object of interest (high enough point cloud density) can be obtained.

In one embodiment, the object of interest may be a region of interest. Here, the region of interest may be a region designated by the user, for example, the user may designate a field of view within a range of 90 degrees in the middle as the region of interest. Then, when acquiring the target point cloud point from other point cloud frames, the point cloud point whose position matches the region of interest in the other point cloud frame can be used as the target point cloud point for adding the multiple points in the cloud. The point cloud points whose positions match the region of interest may at least include point cloud points located within the region of interest. Threshold point cloud points.

In one embodiment, the region of interest may also be the region where the object of interest is located, then when acquiring the target point cloud point from other point cloud frames, the object of interest in the other point cloud frames may be The corresponding point cloud point is used as the target point cloud point for adding to the plurality of point cloud points.

In one embodiment, a plurality of point cloud points collected within the current frame duration may be acquired; the area corresponding to the object of interest in the plurality of point cloud points may be determined; the object of interest may be acquired from other point cloud frames The corresponding point cloud point or the point cloud point corresponding to the area is added to the plurality of point cloud points to obtain the target point cloud frame.

Considering that among the multiple point cloud points collected in the current frame duration, the point cloud points corresponding to the object of interest may be relatively sparse, so the area corresponding to the object of interest may not be determined, or the determined area may not be accurate. Therefore, In one embodiment, the region corresponding to the object of interest in the plurality of point cloud points can be obtained by prediction. In the specific prediction, an embodiment can be determined according to the position of the object of interest in the historical point cloud frame. For example, in the previous frame of the current frame, the position of the object of interest is position A, then it can be predicted that the object of interest is also located in position A among the multiple point cloud points collected during the current frame, that is, position A can be corresponding to position A. The area of is determined as the area corresponding to the object of interest. During prediction, in one embodiment, it can also be predicted according to the motion trajectory of the object of interest in the historical point cloud frame. For example, you can refer to Figure 4. For example, the current frame can be the 5th frame, the object of interest can be a pedestrian, the position of the pedestrian in the 3rd frame can be the position A, and the position in the 4th frame can be the position B, because the position B is to the right of position A, it can be determined that the pedestrian's trajectory is moving to the right, so it can be predicted that the pedestrian's position in the multiple point cloud points collected in the current 5th frame is position C, and position C is located at position B. Right.

In one embodiment, other point cloud frames other than the current frame may be historical point cloud frames before the current frame, or may be future point cloud frames after the current frame. If the other point cloud frames are the historical point cloud frames, since the point cloud points of the historical point cloud frames have been collected, the target point cloud points in the historical point cloud frames can be acquired in real time and added to the multiple points collected within the current frame duration. Point cloud points, generate target point cloud frames in real time for output. If the other point cloud frame is the future point cloud frame, after acquiring multiple point cloud points collected within the current frame duration, it is necessary to wait for the point cloud point collection of the future point cloud frame. After the acquisition is completed, the target point cloud points in the future point cloud frames can be acquired and added to the multiple point cloud points, thereby generating the target point cloud frame for output. At this time, the target point cloud frame is output delayed relative to the current frame. It should be noted that the historical point cloud frame may include the historical target point cloud frame, and the future point cloud frame may also include the future target point cloud frame. For example, the previous frame of the current frame may be the previous target point cloud frame, and the current frame The next frame of can be the next target point cloud frame.

When the other point cloud frames include future point cloud frames, in one embodiment, the target point cloud point may be continuously acquired from each future point cloud frame and added to multiple point clouds collected within the current frame duration point. Specifically, the target point cloud point may be acquired from the next frame of the current frame and added to the multiple point cloud points, and the target point cloud point may also be acquired from the next frame of the current frame and added to the multiple point cloud points. For point cloud points, the target point cloud points are continuously obtained from the future point cloud frames and added until the target point cloud frames meet the preset conditions, and the adding of the target point cloud points from the future point cloud frames can be stopped.

For the preset condition corresponding to the target point cloud frame, there may be various setting methods. In one embodiment, the preset condition may be that the integration time of the target point cloud frame reaches a preset time threshold. Here, the integration time of the target point cloud frame is the collection time corresponding to the point cloud points contained in it, For example, when the duration of a frame is 0.1S, if the time threshold is set to 0.2S, the target point cloud frame can include multiple point cloud points collected within the current frame duration and the target point cloud added in the next frame of the current frame. point; if the time threshold is set to 0.3S, the target point cloud frame can include multiple point cloud points collected within the current frame duration, target point cloud points in the next frame of the current frame, and target point cloud points in the next and next frames of the current frame. The target point cloud point, that is, the integration time corresponding to the target point cloud frame is three frames - 0.3S.

In one embodiment, the preset condition may also be that the point cloud density corresponding to the object of interest in the target point cloud frame reaches a preset density threshold. In one embodiment, the preset condition may also be that the number of point cloud points corresponding to the object of interest in the target point cloud frame reaches a preset number threshold. For the density threshold and the quantity threshold, in one embodiment, they may be determined according to the target category corresponding to the object of interest. For example, if the object of interest can be a person, the density threshold or quantity threshold corresponding to the category of people can be determined according to the pre-configured correspondence. For example, the object of interest can be a vehicle, and the vehicle can be determined according to the pre-configured correspondence. The density threshold or quantity threshold corresponding to this category.

The point cloud processing method provided by the embodiments of the present application may have various applications. In one application, it can be used to improve the accuracy of object recognition in UAV driving scenes. For example, people and vehicles can be used as objects of interest. This method can generate target points with high density of point clouds of people and vehicles. Cloud frame, so that the target point cloud frame can be used for more accurate identification of people and vehicles. In one application, it can be used to improve the tracking accuracy of moving objects. For example, the point cloud density of moving objects in the point cloud frame can be increased by this method, so that the movement trajectory of moving objects can be obtained more accurately, and the tracking of moving objects can be carried out. More accurate tracking.

Referring to FIG. 5 below, FIG. 5 is a schematic structural diagram of a point cloud processing apparatus provided by an embodiment of the present application. The apparatus may include: a processor 510 and a memory 520 storing a computer program, the processor implements the following steps when executing the computer program:

Obtain multiple point cloud points collected within the current frame duration;

determining an object of interest in the plurality of point cloud points;

Optionally, the object of interest includes an object of interest, and the target point cloud point is a point cloud point corresponding to the object of interest.

Optionally, the target point cloud point has undergone position correction before adding the multiple point cloud points.

Optionally, when performing position correction on the target point cloud point, the processor is configured to: according to the position of the object of interest in the plurality of point cloud points and the position of the object in the other point cloud frames. position, and perform position correction on the target point cloud point.

Optionally, the object of interest includes a region of interest.

Optionally, the region of interest is a region designated by a user, and the target point cloud points at least include point cloud points located in the region of interest.

Optionally, the target point cloud point further includes a point cloud point whose distance from the region of interest is less than a preset threshold.

Optionally, the region of interest in the multiple point cloud points is the region where the predicted object of interest is located.

Optionally, the region of interest in the multiple point cloud points is determined according to the position of the object of interest in the historical point cloud frame.

Optionally, the region of interest in the multiple point cloud points is predicted and obtained according to the motion trajectory of the object of interest in the historical point cloud frame.

Optionally, when determining the object of interest in the multiple point cloud points, the processor is configured to determine the multiple point cloud points according to the category corresponding to each point cloud point in the multiple point cloud points. Object of interest in point.

Optionally, the category corresponding to each point cloud point in the multiple point cloud points is obtained by semantically segmenting the multiple point cloud points.

Optionally, the categories corresponding to the point cloud points include one or more of the following: a motion state category, an attribute category, a structure category, and a size category.

Optionally, the motion state category includes one or more of the following: motion state, stationary state, motion direction, and motion speed.

Optionally, the attribute categories include one or more of the following: people, animals, vehicles, buildings, plants, roadblocks, ground, and sky.

Optionally, the other point cloud frames include historical point cloud frames before the current frame, and/or future point cloud frames after the multiple point cloud points.

Optionally, the other point cloud frames include the historical point cloud frames, and the target point cloud frames are output in real time.

Optionally, the other point cloud frames include the future point cloud frames, and the target point cloud frames are output with a delay.

Optionally, when the processor acquires the target point cloud point from other point cloud frames and adds the multiple point cloud points, for each future point cloud frame after the multiple point cloud points, The target point cloud points are acquired and added to the plurality of point cloud points until the target point cloud frame satisfies a preset condition.

Optionally, the preset condition includes that the integration time of the target point cloud frame reaches a preset time threshold.

Optionally, the preset condition includes that the point cloud density of the object of interest in the target point cloud frame reaches a preset density threshold.

Optionally, the target point cloud frame is used for output display.

Optionally, the target point cloud frame is used for an algorithm to perform one or more of the following processing: object detection, semantic segmentation, and target tracking.

The point cloud processing apparatuses of various embodiments are provided above, and for the specific implementation, reference may be made to the corresponding descriptions above, which will not be repeated here.

The point cloud processing device provided by the embodiment of the present application can obtain the target point cloud points associated with the object of interest from other point cloud frames other than the current frame, and add them to the multiple point cloud points collected within the current frame duration, thereby obtaining In the target point cloud frame, there can be more point cloud points corresponding to the object of interest, and the density of the point cloud corresponding to the object of interest can also be higher. By outputting and displaying the target point cloud frame, the user can obtain more information about the object of interest, and processing the target point cloud frame algorithmically can also improve the processing effect of the algorithm, such as improving the recognition accuracy of the object of interest. .

Referring to FIG. 6 below, FIG. 6 is a schematic structural diagram of a ranging apparatus provided by an embodiment of the present application. In one embodiment, the device may be a lidar, and the device may include:

a transmitter 610 for emitting light pulses;

a detector 620 for detecting the light beam reflected by the light pulse;

A processor 630 and a memory 640 storing a computer program that, when executing the computer program, implements the following steps:

Obtain multiple point cloud points collected within the current frame duration;

determining an object of interest in the plurality of point cloud points;

Optionally, the object of interest includes a region of interest.

Optionally, the target point cloud frame is used for output display.

The distance measuring apparatuses of various embodiments are provided above, and reference may be made to the corresponding descriptions in the foregoing for the specific implementation, which will not be repeated here.

The ranging device provided by the embodiment of the present application can obtain the target point cloud points associated with the object of interest from other point cloud frames other than the current frame, and add them to multiple point cloud points collected within the current frame duration, thereby obtaining the target point cloud point. In the point cloud frame, there can be more point cloud points corresponding to the object of interest, and the density of the point cloud corresponding to the object of interest can also be higher. By outputting and displaying the target point cloud frame, the user can obtain more information about the object of interest, and processing the target point cloud frame algorithmically can also improve the processing effect of the algorithm, such as improving the recognition accuracy of the object of interest. .

Referring to FIG. 7 below, FIG. 7 is a schematic structural diagram of a movable platform provided by an embodiment of the present application. The movable platform can be an unmanned vehicle, an unmanned aerial vehicle, an unmanned ship, etc. The movable platform can include:

body 710;

a drive device 720 connected to the body 710 for providing power for the movable platform;

The ranging device 730 mounted on the body 710 is used to implement the following steps:

Obtain multiple point cloud points collected within the current frame duration;

determining an object of interest in the plurality of point cloud points;

Optionally, when the ranging device performs position correction on the target point cloud point, according to the position of the object of interest in the plurality of point cloud points and the position of the object in the other point cloud frames. The position of the target point cloud point is corrected.

Optionally, the object of interest includes a region of interest.

Optionally, when determining the object of interest in the plurality of point cloud points, the ranging device is configured to determine the plurality of points according to the category corresponding to each point cloud point in the plurality of point cloud points. Objects of interest in cloud points.

Optionally, when the distance measuring device acquires the target point cloud point from other point cloud frames and adds the multiple point cloud points, it is used to measure each future point cloud frame after the multiple point cloud points. , the target point cloud points are acquired and added to the plurality of point cloud points until the target point cloud frame satisfies the preset conditions.

Optionally, the target point cloud frame is used for output display.

The movable platforms of various embodiments are provided above, and for the specific implementation, reference may be made to the corresponding descriptions above, which will not be repeated here.

The movable platform provided by the embodiment of the present application can obtain the target point cloud points associated with the object of interest from other point cloud frames other than the current frame, and add them to the multiple point cloud points collected during the current frame duration, so as to obtain the target point cloud point. In the point cloud frame, there can be more point cloud points corresponding to the object of interest, and the density of the point cloud corresponding to the object of interest can also be higher. By outputting and displaying the target point cloud frame, the user can obtain more information about the object of interest, and processing the target point cloud frame algorithmically can also improve the processing effect of the algorithm, such as improving the recognition accuracy of the object of interest. .

Embodiments of the present application further provide a computer-readable storage medium, where the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, implements the point cloud processing method provided by the embodiments of the present application.

Various implementations are provided above for each protection subject. On the basis of no conflict or contradiction, those skilled in the art can freely combine various implementations according to the actual situation, thereby forming various technical solutions. . However, this application document is limited in space and cannot describe all the technical solutions obtained by combination, but it can be understood that these technical solutions that cannot be developed also belong to the scope disclosed in the embodiments of this application.

Embodiments of the present application may take the form of a computer program product implemented on one or more storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having program code embodied therein. Computer-usable storage media includes permanent and non-permanent, removable and non-removable media, and storage of information can be accomplished by any method or technology. Information may be computer readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), Flash Memory or other memory technology, Compact Disc Read Only Memory (CD-ROM), Digital Versatile Disc (DVD) or other optical storage, Magnetic tape cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission medium that can be used to store information that can be accessed by a computing device.

It should be noted that, in this document, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any relationship between these entities or operations. any such actual relationship or sequence exists. The terms "comprising", "comprising" or any other variation thereof are intended to encompass non-exclusive inclusion such that a process, method, article or device comprising a list of elements includes not only those elements, but also other not expressly listed elements, or also include elements inherent to such a process, method, article or apparatus. Without further limitation, an element qualified by the phrase "comprising a..." does not preclude the presence of additional identical elements in a process, method, article or apparatus that includes the element.

The methods and devices provided by the embodiments of the present invention have been described in detail above. The principles and implementations of the present invention are described with specific examples in this paper. The descriptions of the above embodiments are only used to help understand the methods of the present invention and its implementation. At the same time, for those of ordinary skill in the art, according to the idea of the present invention, there will be changes in the specific implementation and application scope. To sum up, the content of this description should not be construed as a limitation to the present invention. .

Claims

A point cloud processing method, comprising:

Obtain multiple point cloud points collected within the current frame duration;

determining an object of interest in the plurality of point cloud points;

A target point cloud point associated with the object of interest is acquired from other point cloud frames other than the current frame, and added to the plurality of point cloud points to obtain a target point cloud frame.
The method according to claim 1, wherein the object of interest comprises an object of interest, and the target point cloud point is a point cloud point corresponding to the object of interest.
The method according to claim 2, wherein the target point cloud point has undergone position correction before adding the plurality of point cloud points.
The method according to claim 3, wherein performing position correction on the target point cloud point comprises:

Position correction is performed on the target point cloud point according to the position of the object of interest in the plurality of point cloud points and the position in the other point cloud frames.
The method of claim 1, wherein the object of interest comprises a region of interest.
The method according to claim 5, wherein the region of interest is a region designated by a user, and the target point cloud points at least include point cloud points located in the region of interest.
The method according to claim 6, wherein the target point cloud point further comprises a point cloud point whose distance from the region of interest is less than a preset threshold.
The method according to claim 5, wherein the region of interest in the plurality of point cloud points is the region where the predicted object of interest is located.
The method according to claim 8, wherein the region of interest in the plurality of point cloud points is determined according to the position of the object of interest in a historical point cloud frame.
The method according to claim 8, wherein the region of interest in the plurality of point cloud points is predicted according to the motion trajectory of the object of interest in historical point cloud frames.
The method according to claim 1, wherein the determining the object of interest in the plurality of point cloud points comprises:

An object of interest in the plurality of point cloud points is determined according to the category corresponding to each point cloud point in the plurality of point cloud points.
The method according to claim 11, wherein the category corresponding to each point cloud point in the plurality of point cloud points is obtained by semantically segmenting the plurality of point cloud points.
The method according to claim 11, wherein the categories corresponding to the point cloud points include one or more of the following: a motion state category, an attribute category, a structure category, and a size category.
The method according to claim 13, wherein the motion state categories include one or more of the following: motion state, stationary state, motion direction, and motion speed.
The method of claim 13, wherein the attribute categories include one or more of the following: people, animals, vehicles, buildings, plants, roadblocks, ground, and sky.
The method according to claim 1, wherein the other point cloud frames include historical point cloud frames before the current frame, and/or future point cloud frames after the plurality of point cloud points.
The method according to claim 16, wherein the other point cloud frames include the historical point cloud frames, and the target point cloud frames are output in real time.
The method according to claim 16, wherein the other point cloud frames include the future point cloud frames, and the target point cloud frames are output with a delay.
The method according to claim 18, wherein acquiring the target point cloud points from other point cloud frames and adding them to the plurality of point cloud points comprises:

For each future point cloud frame after the plurality of point cloud points, the target point cloud point therein is acquired and added to the plurality of point cloud points, until the target point cloud frame meets the preset condition.
The method according to claim 19, wherein the preset condition comprises that the integration time of the target point cloud frame reaches a preset time threshold.
The method according to claim 19, wherein the preset condition comprises that the point cloud density of the object of interest in the target point cloud frame reaches a preset density threshold.
The method according to claim 1, wherein the target point cloud frame is used for output display.
The method according to claim 1, wherein the target point cloud frame is used for an algorithm to perform one or more of the following processing: object detection, semantic segmentation, and target tracking.
A point cloud processing device, comprising: a processor and a memory storing a computer program, wherein the processor implements the following steps when executing the computer program:

Obtain multiple point cloud points collected within the current frame duration;

determining an object of interest in the plurality of point cloud points;

A target point cloud point associated with the object of interest is acquired from other point cloud frames other than the current frame, and added to the plurality of point cloud points to obtain a target point cloud frame.
The apparatus according to claim 24, wherein the object of interest comprises an object of interest, and the target point cloud point is a point cloud point corresponding to the object of interest.
The device according to claim 25, wherein the target point cloud point has undergone position correction before adding the plurality of point cloud points.
The device according to claim 26, wherein when the processor performs position correction on the target point cloud point, the processor is configured to, according to the position of the object of interest in the plurality of point cloud points and the At the positions in the other point cloud frames, position correction is performed on the target point cloud points.
The apparatus of claim 24, wherein the object of interest comprises a region of interest.
The apparatus according to claim 28, wherein the region of interest is a region designated by a user, and the target point cloud points at least include point cloud points located in the region of interest.
The device according to claim 29, wherein the target point cloud point further comprises a point cloud point whose distance from the region of interest is less than a preset threshold.
The apparatus according to claim 28, wherein the region of interest in the plurality of point cloud points is the region where the predicted object of interest is located.
The apparatus according to claim 31, wherein the region of interest in the plurality of point cloud points is determined according to the position of the object of interest in a historical point cloud frame.
The apparatus according to claim 31, wherein the region of interest in the plurality of point cloud points is predicted according to the motion trajectory of the object of interest in historical point cloud frames.
The apparatus according to claim 24, wherein when determining the object of interest in the plurality of point cloud points, the processor is configured to: category, to determine the object of interest in the plurality of point cloud points.
The device according to claim 34, wherein the category corresponding to each point cloud point in the plurality of point cloud points is obtained by semantically segmenting the plurality of point cloud points.
The apparatus according to claim 34, wherein the categories corresponding to the point cloud points include one or more of the following: a motion state category, an attribute category, a structure category, and a size category.
The device according to claim 36, wherein the motion state categories include one or more of the following: motion state, rest state, motion direction, and motion speed.
The apparatus of claim 36, wherein the attribute categories include one or more of the following: people, animals, vehicles, buildings, plants, roadblocks, ground, sky.
The apparatus according to claim 24, wherein the other point cloud frames include historical point cloud frames before the current frame, and/or future point cloud frames after the plurality of point cloud points.
The apparatus according to claim 39, wherein the other point cloud frames include the historical point cloud frames, and the target point cloud frames are output in real time.
The apparatus according to claim 39, wherein the other point cloud frames include the future point cloud frames, and the target point cloud frames are output with a delay.
The device according to claim 41, wherein when the processor acquires the target point cloud point from other point cloud frames and adds the multiple point cloud points For each future point cloud frame after that, the target point cloud point is acquired and added to the plurality of point cloud points until the target point cloud frame satisfies the preset condition.
The device according to claim 42, wherein the preset condition comprises that the integration time of the target point cloud frame reaches a preset time threshold.
The device according to claim 42, wherein the preset condition comprises that the point cloud density of the object of interest in the target point cloud frame reaches a preset density threshold.
The apparatus according to claim 24, wherein the target point cloud frame is used for output display.
The apparatus according to claim 24, wherein the target point cloud frame is used for an algorithm to perform one or more of the following processing: object detection, semantic segmentation, and target tracking.
A distance measuring device, comprising:

an emitter for emitting light pulses;

a detector for detecting the light beam reflected by the light pulse;

A processor and a memory in which a computer program is stored, the processor implementing the following steps when executing the computer program:

Obtain multiple point cloud points collected within the current frame duration;

determining an object of interest in the plurality of point cloud points;

A target point cloud point associated with the object of interest is acquired from other point cloud frames other than the current frame, and added to the plurality of point cloud points to obtain a target point cloud frame.
The device according to claim 47, wherein the object of interest comprises an object of interest, and the target point cloud point is a point cloud point corresponding to the object of interest.
The device according to claim 48, wherein the target point cloud point is subjected to position correction before adding the plurality of point cloud points.
The device according to claim 49, wherein when the processor performs position correction on the target point cloud point, the processor is configured to: according to the position of the object of interest in the plurality of point cloud points and the At the positions in the other point cloud frames, position correction is performed on the target point cloud points.
48. The apparatus of claim 47, wherein the object of interest comprises a region of interest.
The device according to claim 51, wherein the region of interest is a region designated by a user, and the target point cloud points at least include point cloud points located in the region of interest.
The device according to claim 52, wherein the target point cloud point further comprises a point cloud point whose distance from the region of interest is less than a preset threshold.
The apparatus according to claim 51, wherein the region of interest in the plurality of point cloud points is the region where the predicted object of interest is located.
The apparatus according to claim 54, wherein the region of interest in the plurality of point cloud points is determined according to the position of the object of interest in a historical point cloud frame.
The device according to claim 54, wherein the region of interest in the plurality of point cloud points is predicted according to the motion trajectory of the object of interest in historical point cloud frames.
The apparatus according to claim 47, wherein when determining the object of interest in the plurality of point cloud points, the processor is configured to: category, to determine the object of interest in the plurality of point cloud points.
The device according to claim 57, wherein the category corresponding to each point cloud point in the plurality of point cloud points is obtained by semantically segmenting the plurality of point cloud points.
The apparatus according to claim 57, wherein the categories corresponding to the point cloud points include one or more of the following: a motion state category, an attribute category, a structure category, and a size category.
The device according to claim 59, wherein the motion state categories include one or more of the following: motion state, rest state, motion direction, and motion speed.
The apparatus of claim 59, wherein the attribute categories include one or more of the following: people, animals, vehicles, buildings, plants, roadblocks, ground, and sky.
The apparatus according to claim 47, wherein the other point cloud frames include historical point cloud frames before the current frame, and/or future point cloud frames after the plurality of point cloud points.
The apparatus according to claim 62, wherein the other point cloud frames include the historical point cloud frames, and the target point cloud frames are output in real time.
The apparatus according to claim 62, wherein the other point cloud frames include the future point cloud frames, and the target point cloud frames are output with a delay.
The apparatus according to claim 64, wherein, when the processor acquires the target point cloud point from other point cloud frames and adds the multiple point cloud points, the processor is used to: For each future point cloud frame after that, the target point cloud point is acquired and added to the plurality of point cloud points until the target point cloud frame satisfies the preset condition.
The device according to claim 65, wherein the preset condition comprises that the integration time of the target point cloud frame reaches a preset time threshold.
The device according to claim 65, wherein the preset condition comprises that the point cloud density of the object of interest in the target point cloud frame reaches a preset density threshold.
The apparatus of claim 47, wherein the target point cloud frame is used for output display.
The device according to claim 47, wherein the target point cloud frame is used for an algorithm to perform one or more of the following processing: object detection, semantic segmentation, and target tracking.
A movable platform, characterized in that, comprising:

body;

a drive device connected to the body;

A ranging device mounted on the body, the ranging device is used to implement the following steps:

Obtain multiple point cloud points collected within the current frame duration;

determining an object of interest in the plurality of point cloud points;

A target point cloud point associated with the object of interest is acquired from other point cloud frames other than the current frame, and added to the plurality of point cloud points to obtain a target point cloud frame.
The movable platform according to claim 70, wherein the object of interest comprises an object of interest, and the target point cloud point is a point cloud point corresponding to the object of interest.
The movable platform according to claim 71, wherein the target point cloud point has undergone position correction before adding the plurality of point cloud points.
The movable platform according to claim 72, wherein when the ranging device performs position correction on the target point cloud point, the distance measurement device is used for, according to the object of interest, in the plurality of point cloud points The position of , and the position in the other point cloud frames, perform position correction on the target point cloud point.
The movable platform of claim 70, wherein the object of interest comprises a region of interest.
The movable platform according to claim 74, wherein the region of interest is a region designated by a user, and the target point cloud points at least include point cloud points located in the region of interest.
The movable platform according to claim 75, wherein the target point cloud point further includes a point cloud point whose distance from the region of interest is less than a preset threshold.
The movable platform of claim 74, wherein the region of interest in the plurality of point cloud points is the region where the predicted object of interest is located.
The movable platform according to claim 77, wherein the region of interest in the plurality of point cloud points is determined according to the position of the object of interest in the historical point cloud frame.
The movable platform according to claim 77, wherein the region of interest in the plurality of point cloud points is predicted according to the motion trajectory of the object of interest in historical point cloud frames.
The movable platform according to claim 70, wherein when the ranging device determines the object of interest in the plurality of point cloud points, according to each point cloud in the plurality of point cloud points The category corresponding to the point is determined, and the object of interest in the multiple point cloud points is determined.
The movable platform according to claim 80, wherein the category corresponding to each point cloud point in the plurality of point cloud points is obtained by semantically segmenting the plurality of point cloud points.
The movable platform according to claim 80, wherein the categories corresponding to the point cloud points include one or more of the following: a motion state category, an attribute category, a structure category, and a size category.
The movable platform according to claim 82, wherein the motion state categories include one or more of the following: motion state, stationary state, motion direction, and motion speed.
The movable platform of claim 82, wherein the attribute categories include one or more of the following: people, animals, vehicles, buildings, plants, roadblocks, ground, sky.
The movable platform according to claim 70, wherein the other point cloud frames include historical point cloud frames before the current frame, and/or future point cloud frames after the plurality of point cloud points .
The movable platform of claim 85, wherein the other point cloud frames include the historical point cloud frames, and the target point cloud frames are output in real time.
The movable platform of claim 85, wherein the other point cloud frames include the future point cloud frames, and the target point cloud frames are output with a delay.
The movable platform according to claim 87, wherein, when the ranging device obtains the target point cloud point from other point cloud frames and adds the multiple point cloud points, it is used to measure the multiple point cloud points. For each future point cloud frame after the point cloud point, the target point cloud point is acquired and added to the plurality of point cloud points until the target point cloud frame satisfies the preset condition.
The movable platform according to claim 88, wherein the preset condition includes that the integration time of the target point cloud frame reaches a preset time threshold.
The movable platform according to claim 88, wherein the preset condition includes that the point cloud density of the object of interest in the target point cloud frame reaches a preset density threshold.
The movable platform of claim 70, wherein the target point cloud frame is used for output display.
The movable platform according to claim 70, wherein the target point cloud frame is used for an algorithm to perform one or more of the following processing: object detection, semantic segmentation, and target tracking.
A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the point cloud processing method according to any one of claims 1-23 is implemented .