WO2022126427A1

WO2022126427A1 - Point cloud processing method, point cloud processing apparatus, mobile platform, and computer storage medium

Info

Publication number: WO2022126427A1
Application number: PCT/CN2020/136819
Authority: WO
Inventors: 夏清; 李延召
Original assignee: 深圳市大疆创新科技有限公司
Priority date: 2020-12-16
Filing date: 2020-12-16
Publication date: 2022-06-23
Also published as: CN114556427A

Abstract

A point cloud processing method, a point cloud processing apparatus, a mobile platform, and a computer storage medium. The point cloud processing method comprises: obtaining point cloud data acquired by a ranging device and point cloud density distribution data of the ranging device (S501), the point cloud density distribution data being related to the scanning mode of the ranging device, the point cloud density distribution data comprising a plurality of significance coefficients, and the plurality of significance coefficients being used for characterizing distribution characteristics of mapping points of the point cloud data on a reference surface; determining a predetermined processing mode of the point cloud data according to the significance coefficients corresponding to the mapping points of the point cloud data on the reference surface (S502), the predetermined processing mode comprising at least one of a first processing mode and a second processing mode, the first processing mode being used for increasing the point cloud density in at least part of the area of the point cloud data, and the second processing mode being used for reducing the point cloud density in at least part of the area of the point cloud data; and processing the point cloud data according to the predetermined processing mode (S503).

Description

Point cloud processing method, point cloud processing device, movable platform and computer storage medium

manual

technical field

The present invention generally relates to the technical field of ranging devices, and more particularly, to a point cloud processing method, a point cloud processing device, a movable platform and a computer storage medium.

Background technique

In a lidar scanning system, the distribution of the laser point cloud is always spatially inhomogeneous due to the lidar scanning mechanism. Due to the limitation of the lidar optical scanning mechanism, in the generated 3D point cloud data, due to its own line scanning characteristics, there must be an uneven distribution of point clouds on the same plane. Due to the existence of these objective factors, in the same scene, the same object is in the same plane, or the number and density of points in the 3D point cloud obtained by scanning the same object at different distances are different. This difference will lead to subsequent The detection, segmentation, tracking and other algorithms are greatly affected.

SUMMARY OF THE INVENTION

The present invention has been made to solve at least one of the above-mentioned problems. Specifically, one aspect of the present invention provides a point cloud processing method, comprising: acquiring point cloud data collected by a ranging device and point cloud density distribution data of the ranging device, where the point cloud density distribution data is the same as the point cloud density distribution data. The scanning method of the ranging device is related, wherein the point cloud density distribution data includes multiple significance coefficients, and the multiple significance coefficients are used to characterize the distribution characteristics of the point cloud data on the reference plane. the saliency coefficients corresponding to the mapping points of the point cloud data on the reference plane, and determine a predetermined processing method of the point cloud data, wherein the predetermined processing methods include a first processing method and a second processing method At least one of, the first processing method is used to increase the point cloud density in at least a partial area in the point cloud data, and the second processing method is used to reduce the point cloud data in at least a partial area. Cloud density; processing the point cloud data according to the predetermined processing method.

Another aspect of the present invention provides a point cloud processing apparatus. The point cloud processing apparatus includes: a memory for storing executable instructions; a processor for executing the instructions stored in the memory, so that the processor executes The following steps: acquiring point cloud data collected by the ranging device and point cloud density distribution data of the ranging device, where the point cloud density distribution data is related to the scanning mode of the ranging device, wherein the point cloud The density distribution data includes a plurality of significance coefficients, and the plurality of significance coefficients are used to characterize the distribution characteristics of the mapped points of the point cloud data on the reference plane; The corresponding significance coefficient determines a predetermined processing method of the point cloud data, wherein the predetermined processing method includes at least one of a first processing method and a second processing method, and the first processing method is used for increasing the point cloud density in at least a part of the point cloud data, and the second processing method is used to reduce the point cloud density in at least part of the point cloud data; according to the predetermined processing method, Cloud data for processing.

Another aspect of the present invention provides a movable platform, the movable platform includes: a movable platform body, at least one ranging device and the aforementioned point cloud processing device; at least one ranging device is disposed on the movable platform body , which is used to collect point cloud data of the target scene.

Another aspect of the present invention provides a computer storage medium on which a computer program is stored, and when the computer program is executed by a processor, implements the aforementioned point cloud processing method.

According to the point cloud processing method of the embodiment of the present invention, on the premise of keeping the hardware cost unchanged, by acquiring the point cloud density distribution data including a plurality of significance coefficients of the ranging device, according to the point cloud data in the The saliency coefficients corresponding to the mapped points on the surface are used to determine the predetermined processing method of the point cloud data, which can effectively combine the scanning characteristics of the scanning system of the ranging device and the spatial distribution of the point cloud. The cloud data is processed in a suitable manner, thereby effectively improving the distribution uniformity of the point cloud data collected by the ranging device, and can effectively reduce the influence of the point cloud distribution generated by the scanning system on the point cloud density. The point cloud data processed by the method can be used as the basis for subsequent algorithms, so that the subsequent algorithms can be more accurate and reduce processing errors caused by uneven distribution of point clouds, and the method of the present application does not increase hardware costs.

Description of drawings

In order to illustrate the technical solutions in the embodiments of the present invention more clearly, the following briefly introduces the accompanying drawings used in the description of the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can also be obtained from these drawings without creative labor.

FIG. 1 shows a schematic structural diagram of a ranging apparatus in an embodiment of the present invention;

FIG. 2 shows a schematic diagram of a distance measuring device in an embodiment of the present invention;

3 shows a schematic diagram of a scanning pattern of a ranging device in an embodiment of the present invention;

FIG. 4 shows a schematic diagram of a scanning pattern of a distance measuring device in another embodiment of the present invention;

FIG. 5 shows a schematic flowchart of a point cloud processing method in an embodiment of the present invention;

6 shows a schematic diagram of a point cloud distribution saliency map of a ranging device with a first scanning mode in an embodiment of the present invention;

7 shows a schematic diagram of a point cloud distribution saliency map of a ranging device with a second scanning mode according to an embodiment of the present invention;

FIG. 8 shows a schematic block diagram of a point cloud processing apparatus in an embodiment of the present invention;

FIG. 9 shows a schematic block diagram of a movable platform in an embodiment of the present invention.

Detailed ways

In order to make the objects, technical solutions and advantages of the present invention more apparent, exemplary embodiments according to the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only some of the embodiments of the present invention, not all of the embodiments of the present invention, and it should be understood that the present invention is not limited by the example embodiments described herein. Based on the embodiments of the present invention described in the present invention, all other embodiments obtained by those skilled in the art without creative efforts shall fall within the protection scope of the present invention.

In the following description, numerous specific details are set forth in order to provide a more thorough understanding of the present invention. It will be apparent, however, to one skilled in the art that the present invention may be practiced without one or more of these details. In other instances, some technical features known in the art have not been described in order to avoid obscuring the present invention.

It should be understood that the present invention may be embodied in different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used herein, the singular forms "a," "an," and "the/the" are intended to include the plural forms as well, unless the context clearly dictates otherwise. It should also be understood that the terms "compose" and/or "include", when used in this specification, identify the presence of stated features, integers, steps, operations, elements and/or components, but do not exclude one or more other The presence or addition of features, integers, steps, operations, elements, parts and/or groups. As used herein, the term "and/or" includes any and all combinations of the associated listed items.

For a thorough understanding of the present invention, detailed structures will be presented in the following description in order to explain the technical solutions proposed by the present invention. Alternative embodiments of the present invention are described in detail below, however, the invention is capable of other embodiments in addition to these detailed descriptions.

The point cloud processing method, point cloud processing device, movable platform and computer storage medium of the present application will be described in detail below with reference to the accompanying drawings. The features of the embodiments and implementations described below may be combined with each other without conflict.

First, the structure of a ranging device in an embodiment of the present invention is described in detail and exemplarily with reference to FIG. 1 and FIG. 2 . The ranging device includes a laser radar. The ranging device is only used as an example. For other suitable ranging devices Also applicable to this application.

The solutions provided by the various embodiments of the present invention can be applied to a ranging device, and the ranging device can be an electronic device such as a laser radar or a laser ranging device. In one embodiment, the ranging device is used to sense external environmental information, for example, distance information, orientation information, reflection intensity information, speed information and the like of environmental objects. In an implementation manner, the ranging device can detect the distance from the detected object to the ranging device by measuring the time of light propagation between the ranging device and the detected object, that is, Time-of-Flight (TOF). Alternatively, the ranging device can also detect the distance from the detected object to the ranging device through other technologies, such as a ranging method based on phase shift measurement, or a ranging method based on frequency shift measurement. This does not limit.

For ease of understanding, the working process of ranging will be described by way of example below with reference to the ranging apparatus 100 shown in FIG. 1 .

As an example, the ranging apparatus 100 includes a transmitting module, a scanning module and a detection module, the transmitting module is used for transmitting a sequence of optical pulses to detect a target scene; the scanning module is used for sequentially changing the propagation path of the optical pulse sequence transmitted by the transmitting module The detection module is used for receiving the light pulse sequence reflected back by the object, and determining the distance and/or the distance of the object relative to the ranging device according to the reflected light pulse sequence. Orientation to generate the point cloud points. In this paper, the scanning module is also called the scanning system.

Specifically, as shown in FIG. 1 , the transmitting module includes a transmitting circuit 110 ; the detecting module includes a receiving circuit 120 , a sampling circuit 130 and an arithmetic circuit 140 .

The transmit circuit 110 may emit a sequence of light pulses (eg, a sequence of laser pulses). The receiving circuit 120 can receive the optical pulse sequence reflected by the object to be detected, that is, obtain the pulse waveform of the echo signal through it, and perform photoelectric conversion on the optical pulse sequence to obtain an electrical signal, and then process the electrical signal to obtain an electrical signal. output to the sampling circuit 130 . The sampling circuit 130 may sample the electrical signal to obtain a sampling result. The arithmetic circuit 140 may determine the distance, that is, the depth, between the distance measuring device 100 and the detected object based on the sampling result of the sampling circuit 130 .

Optionally, the distance measuring device 100 may further include a control circuit 150, which can control other circuits, for example, can control the working time of each circuit and/or set parameters for each circuit.

It should be understood that although the distance measuring device shown in FIG. 1 includes a transmitting circuit, a receiving circuit, a sampling circuit and an arithmetic circuit for emitting a beam of light for detection, the embodiment of the present application is not limited to this, the transmitting circuit The number of any one of the receiving circuits, sampling circuits, and arithmetic circuits may also be at least two, for emitting at least two light beams in the same direction or in different directions respectively; wherein, the at least two light beam paths can be simultaneously The ejection can also be ejected at different times. In one example, the light-emitting chips in the at least two emission circuits are packaged in the same module. For example, each emitting circuit includes one laser emitting chip, and the dies in the laser emitting chips in the at least two emitting circuits are packaged together and accommodated in the same packaging space.

In some implementations, in addition to the circuit shown in FIG. 1 , the distance measuring device 100 may further include a scanning module for changing the propagation direction of at least one optical pulse sequence (eg, a laser pulse sequence) output from the transmitting circuit to output the field of view. to scan. Exemplarily, the scanning area of the scanning module within the field of view of the ranging device increases over time.

Wherein, the module including the transmitting circuit 110, the receiving circuit 120, the sampling circuit 130 and the operation circuit 140, or the module including the transmitting circuit 110, the receiving circuit 120, the sampling circuit 130, the operation circuit 140 and the control circuit 150 may be referred to as the measuring circuit A ranging module, which can be independent of other modules, such as a scanning module.

A coaxial optical path may be used in the ranging device, that is, the light beam emitted by the ranging device and the reflected light beam share at least part of the optical path in the ranging device. For example, after at least one laser pulse sequence emitted by the transmitting circuit changes its propagation direction through the scanning module, the laser pulse sequence reflected by the detection object passes through the scanning module and then enters the receiving circuit. Alternatively, the distance-measuring device may also adopt an off-axis optical path, that is, the light beam emitted by the distance-measuring device and the reflected light beam are respectively transmitted along different optical paths in the distance-measuring device. FIG. 2 shows a schematic diagram of an embodiment in which the distance measuring device of the present invention adopts a coaxial optical path.

The ranging apparatus 200 includes a ranging module 210, and the ranging module 210 includes a transmitter 203 (which may include the above-mentioned transmitting circuit), a collimating element 204, a detector 205 (which may include the above-mentioned receiving circuit, sampling circuit and arithmetic circuit) and Optical path changing element 206 . The ranging module 210 is used for emitting a light beam, receiving the returning light, and converting the returning light into an electrical signal. Among them, the transmitter 203 can be used to transmit a sequence of optical pulses. In one embodiment, the transmitter 203 may emit a sequence of laser pulses. Optionally, the laser beam emitted by the transmitter 203 is a narrow bandwidth beam with a wavelength outside the visible light range. The collimating element 204 is disposed on the outgoing light path of the transmitter, and is used for collimating the light beam emitted from the transmitter 203, and collimating the light beam emitted by the transmitter 203 into parallel light and outputting to the scanning module. The collimating element also serves to converge at least a portion of the return light reflected by the probe. The collimating element 204 may be a collimating lens or other elements capable of collimating light beams.

In the embodiment shown in FIG. 2, the transmitting optical path and the receiving optical path in the ranging device are combined by the optical path changing element 206 before the collimating element 204, so that the transmitting optical path and the receiving optical path can share the same collimating element, so that the optical path more compact. In some other implementations, the emitter 203 and the detector 205 may use respective collimating elements, and the optical path changing element 206 may be arranged on the optical path behind the collimating element.

In the embodiment shown in FIG. 2 , since the beam aperture of the beam emitted by the transmitter 203 is small, and the beam aperture of the return light received by the ranging device is relatively large, the optical path changing element can use a small-area reflective mirror to The transmit light path and the receive light path are combined. In some other implementations, the optical path changing element may also use a reflector with a through hole, wherein the through hole is used to transmit the outgoing light of the emitter 203 , and the reflector is used to reflect the return light to the detector 205 . In this way, in the case of using a small reflector, the occlusion of the return light by the support of the small reflector can be reduced.

In the embodiment shown in FIG. 2 , the optical path altering element is offset from the optical axis of the collimating element 204 . In some other implementations, the optical path altering element may also be located on the optical axis of the collimating element 204 .

The ranging device 200 further includes a scanning module 202 . The scanning module 202 is placed on the outgoing optical path of the ranging module 210 . The scanning module 202 is used to change the transmission direction of the collimated beam 219 emitted by the collimating element 204 and project it to the external environment, and project the return light to the collimating element 204 . The returned light is focused on the detector 205 through the collimating element 204 .

In one embodiment, the scanning module 202 may include at least one optical element for changing the propagation path of the light beam, wherein the optical element may change the light beam propagation path by reflecting, refracting, diffracting the light beam, etc., such as The optical element includes at least one light-refractive element having non-parallel exit and entrance surfaces. For example, the scanning module 202 includes lenses, mirrors, prisms, galvanometers, gratings, liquid crystals, optical phased arrays (Optical Phased Array) or any combination of the above optical elements. In one example, at least part of the optical elements are moving, for example, the at least part of the optical elements are driven to move by a driving module, and the moving optical elements can reflect, refract or diffract the light beam to different directions at different times. In some embodiments, the multiple optical elements of the scanning module 202 may be rotated or oscillated about a common axis 209, each rotating or oscillating optical element being used to continuously change the propagation direction of the incident beam. In one embodiment, the plurality of optical elements of the scanning module 202 may rotate at different rotational speeds, or vibrate at different speeds. In another embodiment, at least some of the optical elements of scan module 202 may rotate at substantially the same rotational speed. In some embodiments, the plurality of optical elements of the scanning module may also be rotated about different axes. In some embodiments, the plurality of optical elements of the scanning module may also rotate in the same direction, or rotate in different directions; or vibrate in the same direction, or vibrate in different directions, which are not limited herein.

In one embodiment, the scanning module 202 includes a first optical element 214 and a driver 216 connected to the first optical element 214, and the driver 216 is used to drive the first optical element 214 to rotate around the rotation axis 209, so that the first optical element 214 changes The direction of the collimated beam 219. The first optical element 214 projects the collimated beam 219 in different directions. In one embodiment, the angle between the direction of the collimated light beam 219 changed by the first optical element and the rotation axis 209 changes with the rotation of the first optical element 214 . In one embodiment, the first optical element 214 includes a pair of opposing non-parallel surfaces through which the collimated beam 219 passes. In one embodiment, the first optical element 214 includes a prism whose thickness varies along at least one radial direction. In one embodiment, the first optical element 214 includes a wedge prism that refracts the collimated light beam 219 .

In one embodiment, the scanning module 202 further includes a second optical element 215 , the second optical element 215 rotates around the rotation axis 209 , and the rotation speed of the second optical element 215 is different from the rotation speed of the first optical element 214 . The second optical element 215 is used to change the direction of the light beam projected by the first optical element 214 . In one embodiment, the second optical element 215 is connected to another driver 217, and the driver 217 drives the second optical element 215 to rotate. The first optical element 214 and the second optical element 215 can be driven by the same or different drivers, so that the rotational speed and/or steering of the first optical element 214 and the second optical element 215 are different, thereby projecting the collimated beam 219 into the external space Different directions can scan a larger spatial range. In one embodiment, the controller 218 controls the

drivers

216 and 217 to drive the first optical element 214 and the second optical element 215, respectively. The rotational speeds of the first optical element 214 and the second optical element 215 may be determined according to the area and pattern expected to be scanned in practical applications.

Drives

216 and 217 may include motors or other drives.

In one embodiment, the second optical element 215 includes a pair of opposing non-parallel surfaces through which the light beam passes. In one embodiment, the second optical element 215 comprises a prism whose thickness varies along at least one radial direction. In one embodiment, the second optical element 215 comprises a wedge prism.

In one embodiment, the scanning module 202 further includes a third optical element (not shown) and a driver for driving the movement of the third optical element. Optionally, the third optical element includes a pair of opposing non-parallel surfaces through which the light beam passes. In one embodiment, the third optical element comprises a prism of varying thickness along at least one radial direction. In one embodiment, the third optical element comprises a wedge prism. At least two of the first, second and third optical elements rotate at different rotational speeds and/or rotations.

In one embodiment, the scanning module includes two or three of the light refraction elements sequentially arranged on the outgoing light path of the light pulse sequence. Optionally, at least two of the light refraction elements in the scanning module are rotated during the scanning process to change the direction of the light pulse sequence.

The scanning paths of the scanning module are different at least at some different times. The rotation of each optical element in the scanning module 202 can project light in different directions, such as the direction of the projected light 211 and the direction 213 . space to scan. When the light 211 projected by the scanning module 202 hits the detected object 201 , a part of the light is reflected by the detected object 201 to the distance measuring device 200 in a direction opposite to the projected light 211 . The returning light 212 reflected by the probe 201 passes through the scanning module 202 and then enters the collimating element 204 .

A detector 205 is placed on the same side of the collimating element 204 as the emitter 203, and the detector 205 is used to convert at least part of the return light passing through the collimating element 204 into an electrical signal.

In one embodiment, each optical element is coated with an anti-reflection coating. Optionally, the thickness of the anti-reflection film is equal to or close to the wavelength of the light beam emitted by the emitter 203, which can increase the intensity of the transmitted light beam.

In one embodiment, a filter layer is coated on the surface of an element located on the beam propagation path in the distance measuring device, or a filter is provided on the beam propagation path for transmitting at least the wavelength band of the light beam emitted by the transmitter, Reflects other bands to reduce noise from ambient light to the receiver.

In some embodiments, the transmitter 203 may comprise a laser diode through which laser pulses are emitted on the nanosecond scale. Further, the laser pulse receiving time can be determined, for example, by detecting the rising edge time and/or the falling edge time of the electrical signal pulse to determine the laser pulse receiving time. In this way, the ranging apparatus 200 can calculate the TOF by using the pulse receiving time information and the pulse sending time information, so as to determine the distance from the probe 201 to the ranging apparatus 200 . The distance and orientation detected by the ranging device 200 can be used for remote sensing, obstacle avoidance, mapping, modeling, navigation, and the like.

The point cloud scanning pattern is generated by the design of a scanning system (also referred to as a scanning module in this paper) of a ranging device such as a lidar, and the influencing factors include the scanning frequency, frame rate, motor speed, and speed ratio of the scanning system. Different scanning systems will show different sampling patterns due to motor speed settings. For example, one scanning system obtains a specific petal-shaped scanning pattern, as shown in Figure 3, while another scanning system obtains eye-type scanning patterns. The scanning pattern with dense and sparse sides in the middle is shown in Figure 4. The scanning pattern herein may refer to a pattern formed by the accumulation of scanning trajectories of a light beam within a scanning field of view over a period of time. Under the scanning of the scanning module, after the light beam forms a complete scanning pattern in one scanning period, it starts to form the next complete, same or different scanning pattern in the next scanning period. Due to its own line scanning characteristics (as shown in Figure 3 and Figure 4), there must be uneven distribution of point clouds on the same plane. Due to the existence of these objective factors, in the same scene, the same object is in the same plane, or the number and density of points in the 3D point cloud obtained by scanning the same object at different distances are different. This difference will lead to subsequent The detection, segmentation, tracking and other algorithms are greatly affected.

At the beginning of the design of the lidar scanning system, the balance between the hardware cost and the scanning method will be considered. Therefore, if the scanning system is modified from the hardware level, the hardware cost and development time will be increased, which will significantly improve the product. research and development costs. In practical applications, the point cloud in the three-dimensional space can be interpolated through the software level, but the accuracy and adaptability of the interpolation is poor, which is easy to bring errors.

In view of the above problems, the present application provides a point cloud processing method, the point cloud processing method includes: acquiring point cloud data collected by a ranging device and point cloud density distribution data of the ranging device, the point cloud The density distribution data is related to the scanning mode of the ranging device, wherein the point cloud density distribution data includes a plurality of significance coefficients, and the plurality of significance coefficients are used to characterize the mapping points of the point cloud data on the reference surface according to the saliency coefficients corresponding to the mapped points of the point cloud data on the reference plane, determine a predetermined processing method of the point cloud data, wherein the predetermined processing method includes the first processing method At least one of the method and the second processing method, the first processing method is used to increase the point cloud density in at least a partial area in the point cloud data, and the second processing method is used to reduce the point cloud data in the point cloud data. point cloud density in at least a part of the area; processing the point cloud data according to the predetermined processing manner.

According to the point cloud processing method of the embodiment of the present invention, on the premise of keeping the hardware cost unchanged, by acquiring the point cloud density distribution data including a plurality of significance coefficients of the ranging device, according to the point cloud data in the The saliency coefficients corresponding to the mapped points on the surface are used to determine the predetermined processing method of the point cloud data, which can effectively combine the scanning characteristics of the scanning system of the ranging device and the spatial distribution of the point cloud. The cloud data is processed in a suitable way, so that the distribution of the processed point cloud data is more uniform and reasonable, which can effectively reduce the impact of the point cloud distribution generated by the scanning system on the point cloud density, and better describe the scanning scene. Object information, the point cloud data processed by the method of the present application can be used as the basis for the subsequent algorithm, so that the subsequent algorithm can be more accurate, reduce processing errors caused by uneven distribution of the point cloud, and the method of the present application does not increase hardware cost.

Hereinafter, the point cloud processing method of the present application will be described with reference to FIG. 5 , wherein FIG. 5 shows a schematic flowchart of the point cloud processing method in an embodiment of the present application.

As an example, the point cloud processing method of the embodiment of the present application includes the following steps S501 to S503:

First, in step S501, the point cloud data collected by the ranging device and the point cloud density distribution data of the ranging device are acquired.

The point cloud density distribution data is related to the scanning mode of the ranging device, wherein the point cloud density distribution data includes multiple significance coefficients, and the multiple significance coefficients are used to characterize the point cloud data on the reference surface. The distribution characteristics of the mapped points on . The reference plane can be any suitable plane, for example, the reference plane is a plane perpendicular to the central axis of the light pulse sequence emitted by the ranging device.

Different scanning patterns will generate corresponding three-dimensional point cloud distribution in three-dimensional space, and the distribution characteristics are mainly reflected in the density distribution of the point cloud and the shape distribution of the point cloud. For non-repetitive scanning ranging devices such as lidar, the spatial distribution of point clouds is dynamic and the density is not uniform.

In one example, acquiring the point cloud data collected by the ranging device and the point cloud density distribution data of the ranging device includes: acquiring at least one frame of scanning pattern corresponding to the scanning mode of the ranging device, according to The number of mapping points on the scanning pattern in different statistical regions determines the significance coefficient of the point cloud corresponding to the mapping points in each statistical region. For example, one frame of point cloud data can be selected, wherein the scanning pattern It can be used to characterize the mapping points of the three-dimensional point cloud data of the ranging device on the reference surface. Optionally, the scanning pattern can be composed of the mapping points of the point cloud data of the ranging device on the reference surface. Usually the statistical area The greater the number of internal mapping points, the greater the density of points in the statistical area. For example, by a ranging device with a first scanning method, the corresponding scanning pattern is shown in Figure 3, and a ranging device with a second scanning method is used. The corresponding scanning pattern of the device is shown in Figure 4. According to the scanning pattern, the distribution characteristics of the 3D point cloud on the reference surface can be obtained.

The scanning pattern is obtained by mapping point cloud data (such as a three-dimensional point cloud) output by the ranging device to the reference surface, for example, a three-dimensional point cloud output by the ranging device scanning a target scene in a three-dimensional space The distribution is obtained by mapping the distribution to the reference surface, or the scanning pattern is obtained by fitting a function according to the scanning mode of the ranging device. Optionally, point cloud data of, for example, 10 Hz, or point cloud data of other suitable frame rates may be selected for scanning pattern selection.

Scanning patterns based on point clouds can extract pattern features, such as extracting density features, such as the non-repeatability and scanning characteristics of lidar scanning systems. The distribution of point clouds in three-dimensional space is uneven and has obvious density distribution characteristics. For example, the point cloud distribution of the scanning pattern shown in Figure 3 is generally circular, with dense middle density and sparse edges, while the point cloud distribution of the VT scanning pattern shown in Figure 3 is approximately rectangular: the central area of the density distribution is dense , sparse on both sides. Therefore, the characteristics of the scanning pattern can be well described by the distribution characteristics of the density in space.

In an example, determining the significance coefficient corresponding to each statistical region according to the number of the mapping points on the scanning pattern in different statistical regions includes: normalizing the number of mapping points in the plurality of statistical regions to obtain the significance coefficients of the point clouds corresponding to the mapped points in each statistical area, for example, uniformly map the quantities in the statistical area to the [0, 1] interval, wherein any number well-known to those skilled in the art can be used. The normalization method is to perform normalization processing on the number of mapping points in a plurality of the statistical regions. Through the normalization processing, the comparison of the quantities of different magnitudes can be facilitated.

The size of the statistical area is different, and different point cloud density distribution data can be obtained. The size of the statistical area can be determined based on any suitable rules. For example, the size of one or more statistical areas can be determined based on the application scenario of the ranging device and the size of the detection target in the application scenario, and the size of different statistical areas can be determined based on the size of the statistical area. , to obtain different point cloud density distribution data.

In one example, the size of the statistical area is determined based on the size of the target object that the ranging device is intended to detect, wherein the target object includes a first target object and a second target object, and the first target object is larger than the size of the second target object, then the statistical area of the point cloud density distribution data corresponding to the first target object has the first size, and the point cloud density distribution data corresponding to the second target object has the first size. The statistical region has a second size, the first size being larger than the second size. By freely setting the appropriate size of the statistical region according to the user's actual scene for saliency calculation, it can more accurately reflect the characteristics of real objects in the 3D scene, and is more conducive to the application of subsequent detection, segmentation and other algorithms.

For example, the application scenarios of ranging devices such as lidar are located indoors or in parks, etc. In these scenarios, there are mainly people or some small objects. Therefore, the size of the objects that may exist in the indoor scene can be used to determine statistics. The size of the area, wherein the sizes of different statistical areas can be made to correspond to the size of a target object respectively.

For another example, when the application scenario is the scenario of automatic driving of vehicles, in this scenario, the ranging device is usually used for vehicle identification, obstacle identification, etc., the size of various statistical regions can be determined to be used for calculating the significance respectively. The coefficients are used to obtain point cloud density distribution data for the identification of various objects.

In one example, the point cloud density distribution data is presented in the form of a heatmap, wherein different pixel values in the heatmap are used to characterize different significance coefficients. For example, the larger the pixel value is, the larger the saliency coefficient of the representation is, and the higher the density of the corresponding position points is. In a specific example, the heat map includes a first pixel point and a second pixel point, and the first pixel The pixel value of the point is greater than the pixel value of the second pixel point, then the significance coefficient corresponding to the first pixel point is greater than the significance coefficient corresponding to the second pixel point; The smaller the significance coefficient of , the smaller the density of the corresponding position points. For example, the heat map includes a first pixel point and a second pixel point, and the pixel value of the first pixel point is greater than that of the second pixel point. pixel value, the significance coefficient corresponding to the first pixel point is smaller than the significance coefficient corresponding to the second pixel point. Optionally, the pixel value includes a gray value, a color value or a brightness value or other image-related values.

In one example, the reference surface may have a plurality of statistical regions, and the point cloud density distribution data includes at least two types of point cloud density distribution data, wherein different types of point cloud density distribution data have different statistical regions size.

For example, the point cloud density distribution data corresponding to various statistical area sizes as shown in Figure 6, the point cloud density distribution data is a saliency map (that is, a heat map), and the nine maps from the first row to the third row represent the

respective Select

10, 20, 30, 40, 50, 60, 70, 80, 90, 100, 200, 300 unit sizes (for example, it can be in square millimeters, square centimeters or square meters, which can be set reasonably according to actual needs) As a saliency map obtained by counting the distribution density of the point cloud in the area, the greater the brightness, the greater the density of the point cloud at the point location. The gray-scale saliency map can also be colored to obtain a colored saliency map.

For another example, the point cloud density distribution data corresponding to various statistical area sizes as shown in Figure 7, the point cloud density distribution data is a saliency map (that is, a heat map), and the six maps are represented from the first row to the third row. Select 10, 40, 80, 100, 200, 300 unit sizes (for example, it can be in square millimeters, square centimeters or square meters, which can be set reasonably according to actual needs) as the significance obtained by the distribution density of point clouds in the statistical area. In the figure, the greater the brightness, the greater the density of the point cloud at the point location. The gray-scale saliency map can also be colored to obtain a colored saliency map.

It can be seen from Fig. 6 and Fig. 7 that different scanning methods will generate different saliency maps, and different saliency maps can be obtained with different sizes of the selected statistical regions. Therefore, the application can freely choose the appropriate saliency map according to the actual scene of the user. The saliency coefficient is calculated according to the size of the statistical area, so that the characteristics of the real objects in the 3D scene can be more accurately reflected, which is more conducive to the application of subsequent detection, segmentation and other algorithms.

The reference surface has a plurality of statistical regions, and the point cloud density distribution data includes at least two types of point cloud density distribution data, wherein different types of point cloud density distribution data have different statistical region sizes, that is, each Types of point cloud density distribution data can have significance maps for many different statistical region sizes.

When the ranging device is used to scan the predetermined scene, acquiring point cloud density distribution data corresponding to the point cloud data collected by the ranging device according to the scanning mode of the ranging device, further comprising: according to the scanning method method and the relationship between the size of the target object and the size of the statistical area predetermined to be identified from the point cloud data, to determine the point cloud density distribution data such as a heat map to be used for preprocessing the point cloud data, for example, The point cloud density distribution data to be used for preprocessing the point cloud data is the point cloud density distribution data whose size of the statistical area is less than or equal to the size of the target object, wherein, preferably, the point cloud density distribution data to be used for the preprocessing of the point cloud data The preprocessed point cloud density distribution data is point cloud density distribution data whose size of the statistical area is substantially equal to the size of the target object. With this setting, when the ranging device is used to sense smaller objects, the heat map with the smaller size of the divided statistical area is selected, and when larger objects are sensed, the heat map with the larger size of the divided statistical area is selected. picture. Therefore, when different targets are perceived through point cloud data, point cloud density distribution data such as heat maps with different statistical area sizes can be used accordingly.

Continuing to refer to FIG. 5 , in step S503, a predetermined processing method of the point cloud data is determined according to the saliency coefficients corresponding to the mapped points of the point cloud data on the reference surface, wherein the predetermined The processing method includes at least one of a first processing method and a second processing method, the first processing method is used to increase the point cloud density in at least a partial area of the point cloud data, and the second processing method is used to Decrease the point cloud density in at least a portion of the point cloud data.

In the above, the point cloud density distribution data to be used, such as a heat map, has been selected according to the target to be perceived. According to the heat map, the significance coefficient corresponding to the point cloud in the point cloud data can be obtained, that is, the point cloud of the point cloud data. The significance coefficient of the corresponding point in the heat map (that is, the corresponding mapping point on the reference surface), exemplarily, according to the point cloud data corresponding to the mapping point on the reference surface. The significance coefficient is determined, and the predetermined processing mode of the point cloud data is determined, including: according to the depth information represented by the point cloud in the point cloud data and the corresponding significance coefficient, determining the space of the point cloud in the point cloud data. Distribution attribute; according to the spatial distribution attribute, determine the predetermined processing method of the point cloud data. By combining the saliency coefficient with the actual position information of the 3D point cloud in space, such as depth information, the attributes of the point cloud in the 3D space can be effectively expressed. Optionally, the depth information includes, for example, the horizontal distance of the point cloud in the three-dimensional point cloud data from the position of the scanning system.

After calculating the saliency maps of different scanning systems, it is necessary to further calculate the saliency attributes of discrete points in the 3D point cloud according to the actual scene. When distant objects are at the junction, there is actually a distance difference between different objects. By combining the saliency coefficient and depth information, the saliency coefficients of points that do not belong to the same object can be distinguished. Specifically, assuming that the spatial coordinates of the ith point in the three-dimensional space are (x _i , y _i , z _i ), then the saliency attribute (that is, the spatial distribution attribute) of the point can be calculated by the following formula:

P _i =S(y _i , z _i )*x _i (1)

where S(y _i , z _i ) represents the saliency coefficient of the 3D point cloud at (y _i , z _i ) on the 2D projected YOZ plane (that is, the reference plane), and _xi represents the level of the 3D point from the position of the scanning system distance.

Formula (1) can effectively combine the saliency map calculated above with the actual position of the three-dimensional point cloud in space, and can effectively express the attributes of the point cloud in three-dimensional space. The above formula can also be reasonably adjusted as required, for example, the saliency coefficient and the horizontal distance between the three-dimensional point and the position of the scanning system can be divided, so as to obtain the spatial distribution attribute.

In the application of target detection and segmentation of 3D point clouds, the spatial coordinates and reflectivity information of point clouds are usually used. These information are less informative for detection or segmentation algorithms, which are easy to cause false detection and require huge amounts of data. . The spatial distribution attributes of the 3D point cloud can be calculated by the aforementioned method. For subsequent segmentation and detection algorithms, it is equivalent to adding a one-dimensional feature output, which can more accurately represent the deep characteristics of the original data.

In addition, the saliency attribute calculated by the present invention can not only be used as the one-dimensional feature input of the algorithm, but also can be used as a reference for algorithms such as detection and segmentation. The smaller the place, the sparser the distribution of the point, then the subsequent algorithm can use this feature to perform selective up-sampling and down-sampling operations on the 3D point cloud in the space, so that the distribution of the point cloud in the space is more regular. For example, when the significance coefficient corresponding to the first part of the point cloud in the point cloud data is within the first threshold range, it can be determined that the predetermined processing method of the first part of the point cloud is the first processing method, and the first processing The method includes one of the following processing methods: interpolation, upsampling, and time accumulation. Through the first processing method, the density of the point cloud can be increased. When the significance coefficient corresponding to the second part of the point cloud in the point cloud data is at the first threshold When it is within the range, it can be determined that the predetermined processing method of the first part of the point cloud is the second processing method. The second processing method includes downsampling, or no processing is performed. The second processing method can reduce the density of the point cloud or Does not change the point cloud density. The first threshold range and the second threshold range may be reasonably set according to actual needs, and are not specifically limited herein.

In other examples, determining the predetermined processing method of the point cloud data according to the spatial distribution attribute includes: when the spatial distribution attribute corresponding to the first part of the point cloud in the point cloud data is within a first threshold range , determine that the predetermined processing method of the first part of the point cloud is the first processing method, and the first processing method includes one of the following processing methods: interpolation, upsampling, and time accumulation. Increase the density of the point cloud; when the spatial distribution attribute corresponding to the second part of the point cloud in the point cloud data is within the second threshold range, determine that the predetermined processing method of the second part of the point cloud is the first Two processing modes, the second processing mode includes downsampling, or no processing is performed, and the point cloud density can be reduced or not changed through the second processing mode. The first threshold range and the second threshold range may be reasonably set according to actual needs, and are not specifically limited herein.

To sum up, according to the point cloud processing method of the embodiment of the present invention, on the premise of keeping the hardware cost unchanged, by acquiring the point cloud density distribution data including a plurality of significance coefficients of the ranging device, according to the point cloud density distribution data. The saliency coefficient corresponding to the mapping point of the cloud data on the reference plane determines the predetermined processing method of the point cloud data, which can effectively combine the scanning characteristics of the scanning system of the ranging device and the spatial distribution of the point cloud, The spatially distributed point cloud data is processed in a suitable way, so that the processed point cloud data distribution is more uniform and reasonable, which can effectively reduce the impact of the point cloud distribution generated by the scanning system on the point cloud density, and better. The object information in the scanned scene is described, and the point cloud data processed by the method of the present application can be used as the basis of the subsequent algorithm, so that the subsequent algorithm can be more accurate, and the processing errors caused by the uneven distribution of the point cloud can be reduced. The method does not increase the hardware cost.

In addition, the method of the present application also combines the saliency coefficient with the actual spatial position in the three-dimensional scene, which can more reasonably and effectively represent the spatial attributes of the three-dimensional point cloud, and can more accurately reflect the characteristics of the real objects in the three-dimensional scene. It is beneficial to the application of subsequent detection, segmentation and other algorithms.

In addition, when the method in the present application calculates the saliency attribute, different statistical region sizes can be selected according to actual needs, so as to adapt to different scenarios.

Next, a point cloud processing apparatus 800 according to an embodiment of the present application is described with reference to FIG. 8 , wherein FIG. 8 shows a schematic block diagram of the point cloud processing apparatus in an embodiment of the present invention.

In some embodiments, as shown in FIG. 8 , the point cloud processing apparatus 800 further includes one or more processors 802 and one or more memories 801 , and the one or more processors 802 work together or individually. Optionally, the point cloud processing device may further include at least one of an input device (not shown), an output device (not shown) and an image sensor (not shown), and these components are connected through a bus system and/or other forms of A connection mechanism (not shown) interconnects.

The memory 801 is used for storing program instructions executable by the processor, for example, for storing corresponding steps and program instructions for implementing the point cloud processing method according to the embodiment of the present application. One or more computer program products may be included, which may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and/or cache memory, or the like. The non-volatile memory may include, for example, read only memory (ROM), hard disk, flash memory, and the like.

The input device may be a device used by a user to input instructions, and may include one or more of a keyboard, a mouse, a microphone, a touch screen, and the like.

The output device can output various information (such as image or sound) to the outside (such as a user), and can include one or more of a display, a speaker, etc., for outputting the processed point cloud as an image or a video, Can also be used to output the obtained saliency map as an image.

A communication interface (not shown) is used for communication between the point cloud processing apparatus and other devices, including wired or wireless communication. The point cloud processing device can access wireless networks based on communication standards, such as WiFi, 2G, 3G, 4G, 5G, or a combination thereof. In one exemplary embodiment, the communication interface receives broadcast signals or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication interface further includes a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

Processor 802 may be a central processing unit (CPU), graphics processing unit (GPU), application specific integrated circuit (ASIC), field programmable gate array (FPGA), or other form of processing with data processing capabilities and/or instruction execution capabilities unit, and can control other components in the point cloud processing device to perform desired functions. The processor 802 can execute the instructions stored in the memory 801 to execute the point cloud processing method of the embodiments of the present application described herein. For example, a processor can include one or more embedded processors, processor cores, microprocessors, logic circuits, hardware finite state machines (FSMs), digital signal processors (DSPs), or combinations thereof. In this embodiment, the processor includes a Field Programmable Gate Array (FPGA), wherein the arithmetic circuit of the point cloud processing apparatus may be a part of the Field Programmable Gate Array (FPGA).

The point cloud processing device includes one or more processors, working together or individually, a memory for storing program instructions; the processor for executing the program instructions stored in the memory, when the program instructions are executed , the processor is configured to implement the point cloud processing method according to the embodiment of the present application, including: acquiring point cloud data collected by a ranging device and point cloud density distribution data of the ranging device, the point cloud density distribution data It is related to the scanning mode of the distance measuring device, wherein the point cloud density distribution data includes a plurality of significance coefficients, and the plurality of significance coefficients are used to characterize the distribution characteristics of the point cloud data on the reference plane. ; According to the saliency coefficients corresponding to the mapping points of the point cloud data on the reference plane, determine a predetermined processing mode of the point cloud data, wherein the predetermined processing mode includes a first processing mode and a first processing mode At least one of two processing methods, the first processing method is used to increase the point cloud density in at least part of the point cloud data, and the second processing method is used to reduce at least part of the point cloud data. point cloud density in the data; according to the predetermined processing method, the point cloud data is processed. Optionally, the reference plane is a plane perpendicular to the central axis of the light pulse sequence emitted by the ranging device.

In one example, the point cloud density distribution data is presented in the form of a heat map (also referred to herein as a saliency map), wherein different pixel values in the heat map are used to characterize different saliency coefficients. For example, the heat map includes a first pixel point and a second pixel point, and the pixel value of the first pixel point is greater than the pixel value of the second pixel point, then the significance coefficient corresponding to the first pixel point is greater than the significance coefficient corresponding to the second pixel point; or, for another example, the heat map includes a first pixel point and a second pixel point, and the pixel value of the first pixel point is greater than that of the second pixel point , the significance coefficient corresponding to the first pixel point is smaller than the significance coefficient corresponding to the second pixel point. Optionally, the pixel value includes a gray value, a color value or a brightness value, or other values that can characterize the size of the saliency coefficient.

In an example, acquiring point cloud density distribution data corresponding to the point cloud data collected by the ranging device according to the scanning mode of the ranging device includes: acquiring data corresponding to the scanning mode of the ranging device At least one frame of scanning pattern, wherein the scanning pattern is composed of the mapping points of the point cloud data of the ranging device on the reference plane; according to the number of mapping points on the scanning pattern in different statistical regions, Determine the significance coefficient of the point cloud corresponding to the mapped points in each statistical area. Optionally, the scanning pattern is obtained by mapping point cloud data output by the ranging device to the reference surface, or the scanning pattern is obtained by fitting a function according to the scanning mode of the ranging device. And the fitting is obtained.

In an example, determining the significance coefficient corresponding to each statistical region according to the number of the mapping points on the scanning pattern in different statistical regions includes: normalizing the number of mapping points in the plurality of statistical regions To obtain the significance coefficient of the point cloud corresponding to the mapped points in each statistical region.

In one example, the size of the statistical area is determined based on the size of the target object that the ranging device is intended to detect, wherein the target object includes a first target object and a second target object, and the first target object is larger than the size of the second target object, then the statistical area of the point cloud density distribution data corresponding to the first target object has the first size, and the point cloud density distribution data corresponding to the second target object has the first size. The statistical region has a second size, the first size being larger than the second size.

In one example, the reference surface has a plurality of statistical regions, and the point cloud density distribution data includes at least two types of point cloud density distribution data, wherein different types of point cloud density distribution data have different statistical region sizes .

In an example, acquiring the point cloud density distribution data corresponding to the point cloud data collected by the ranging device according to the scanning mode of the ranging device, further comprising: according to the scanning mode and a predetermined method from the point cloud The relationship between the size of the object identified in the data and the size of the statistical area determines the point cloud density distribution data. For example, the point cloud density distribution data is point cloud density distribution data in which the size of the statistical area is smaller than or equal to the size of the target object.

In an example, the determining a predetermined processing manner of the point cloud data according to the saliency coefficients corresponding to the mapped points of the point cloud data on the reference surface includes: according to the point cloud data The depth information represented by the midpoint cloud and the corresponding significance coefficient determine the spatial distribution attribute of the point cloud in the point cloud data; according to the spatial distribution attribute, determine the predetermined processing method of the point cloud data.

Exemplarily, determining the predetermined processing manner of the point cloud data according to the spatial distribution attribute includes: when the spatial distribution attribute corresponding to the first part of the point cloud in the point cloud data is within a first threshold range , determine that the predetermined processing mode of the first part of the point cloud is the first processing mode; when the spatial distribution attribute corresponding to the second part of the point cloud in the point cloud data is within the second threshold range, determine that the The predetermined processing manner of the second partial point cloud is the second processing manner. The first processing manner includes one of the following processing manners: interpolation, upsampling, and time accumulation; and the second processing manner includes downsampling.

In an implementation manner, as shown in FIG. 9 , a movable platform 900 is also provided in this embodiment of the present application. The movable platform 900 may include a movable platform body 901 and at least one ranging device 902 . At least one ranging device The device 902 is arranged on the movable platform body 901 and is used to collect point cloud data of the target scene. For the distance measuring device 902, reference may be made to the distance measuring device 100 and the distance measuring device 200 in the foregoing, and the description is not repeated here.

The distance measuring device 902 can be installed on the movable platform body 901 of the movable platform 900 . The movable platform 900 with the distance measuring device can measure the external environment, for example, measure the distance between the movable platform 900 and obstacles for obstacle avoidance and other purposes, and perform two-dimensional or three-dimensional mapping of the external environment. In certain embodiments, the movable platform 900 includes at least one of an unmanned aerial vehicle, a vehicle, a remote-controlled vehicle, a robot, and a boat. When the ranging device is applied to the unmanned aerial vehicle, the movable platform body 901 is the body of the unmanned aerial vehicle. When the distance measuring device is applied to an automobile, the movable platform body 901 is the body of the automobile. The vehicle may be an autonomous driving vehicle or a semi-autonomous driving vehicle, which is not limited herein. When the distance measuring device is applied to the remote control car, the movable platform body 901 is the body of the remote control car. When the distance measuring device is applied to a robot, the movable platform body 901 is a robot.

Further, the movable platform 900 further includes the above-mentioned point cloud processing apparatus 800, and the description of the point cloud processing transposition 800 can be referred to the above.

Since the point cloud processing apparatus 800 in the embodiment of the present application is used to execute the aforementioned method, and the movable platform includes the point cloud processing apparatus 800, both the point cloud processing apparatus 800 and the movable platform 900 have the same method as the aforementioned point cloud processing method. Same advantages.

In addition, an embodiment of the present application further provides a computer storage medium, on which a computer program is stored. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor may execute the program instructions stored in the memory to implement the functions (implemented by the processor) in the embodiments of the present application described herein and/or other desired functions, such as to perform corresponding steps of the point cloud processing method according to the embodiments of the present application, various application programs and various data may also be stored in the computer-readable storage medium, such as the application Various data used and/or generated by the program, etc.

For example, the computer storage medium may include, for example, a memory card for a smartphone, a storage unit for a tablet computer, a hard disk for a personal computer, read only memory (ROM), erasable programmable read only memory (EPROM), portable compact disk Read only memory (CD-ROM), USB memory, or any combination of the above storage media. The computer-readable storage medium can be any combination of one or more computer-readable storage media. For example, a computer-readable storage medium contains computer-readable program codes for converting point cloud data into two-dimensional images, and/or computer-readable program codes for three-dimensional reconstruction of point cloud data, and the like.

It should be understood that various parts of this application may be implemented in hardware, software, firmware, or a combination thereof. In the above-described embodiments, various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented by any one of the following techniques known in the art, or a combination thereof: discrete with logic gates for implementing logic functions on data signals Logic circuits, application-specific integrated circuits with suitable combinational logic gate circuits, Programmable Gate Array (hereinafter referred to as: PGA), Field Programmable Gate Array (Field Programmable Gate Array; referred to as: FPGA), etc.

Although example embodiments have been described herein with reference to the accompanying drawings, it should be understood that the above-described example embodiments are exemplary only, and are not intended to limit the scope of the application thereto. Various changes and modifications may be made therein by those of ordinary skill in the art without departing from the scope and spirit of the present application. All such changes and modifications are intended to be included within the scope of this application as claimed in the appended claims.

Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Skilled artisans may implement the described functionality using different methods for each particular application, but such implementations should not be considered beyond the scope of this application.

In the several embodiments provided in this application, it should be understood that the disclosed apparatus and method may be implemented in other manners. For example, the device embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined or May be integrated into another device, or some features may be omitted, or not implemented.

In the description provided herein, numerous specific details are set forth. It will be understood, however, that the embodiments of the present application may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.

Similarly, it is to be understood that in the description of the exemplary embodiments of the present application, various features of the present application are sometimes grouped together into a single embodiment, FIG. , or in its description. However, this method of application should not be construed as reflecting an intention that the claimed application requires more features than are expressly recited in each claim. Rather, as the corresponding claims reflect, the invention lies in the fact that the corresponding technical problem may be solved with less than all features of a single disclosed embodiment. Thus, the claims following the Detailed Description are hereby expressly incorporated into this Detailed Description, with each claim standing on its own as a separate embodiment of this application.

It will be understood by those skilled in the art that all features disclosed in this specification (including the accompanying claims, abstract and drawings) and any method or apparatus so disclosed may be used in any combination, except that the features are mutually exclusive. Processes or units are combined. Each feature disclosed in this specification (including accompanying claims, abstract and drawings) may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.

Furthermore, those skilled in the art will appreciate that although some of the embodiments described herein include certain features, but not others, included in other embodiments, that combinations of features of different embodiments are intended to be within the scope of the present application within and form different embodiments. For example, in the claims, any of the claimed embodiments may be used in any combination.

Various component embodiments of the present application may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art should understand that a microprocessor or a digital signal processor (DSP) may be used in practice to implement some or all functions of some modules according to the embodiments of the present application. The present application can also be implemented as a program of apparatus (eg, computer programs and computer program products) for performing part or all of the methods described herein. Such a program implementing the present application may be stored on a computer-readable medium, or may be in the form of one or more signals. Such signals may be downloaded from Internet sites, or provided on carrier signals, or in any other form.

It should be noted that the above-described embodiments illustrate rather than limit the application, and alternative embodiments may be devised by those skilled in the art without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The application can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means may be embodied by one and the same item of hardware. The use of the words first, second, and third, etc. do not denote any order. These words can be interpreted as names.

Claims

A point cloud processing method, characterized in that the method comprises:

Acquiring point cloud data collected by the ranging device and point cloud density distribution data of the ranging device, where the point cloud density distribution data is related to the scanning mode of the ranging device, wherein the point cloud density distribution data Including a plurality of significance coefficients, the plurality of significance coefficients are used to characterize the distribution characteristics of the mapping points of the point cloud data on the reference plane;

According to the saliency coefficients corresponding to the mapping points of the point cloud data on the reference plane, a predetermined processing method of the point cloud data is determined, wherein the predetermined processing method includes a first processing method and a second processing method At least one of the processing methods, the first processing method is used to increase the point cloud density in at least part of the area of the point cloud data, and the second processing method is used to reduce the density of the point cloud in at least part of the area in the point cloud data the point cloud density;

According to the predetermined processing manner, the point cloud data is processed.
The point cloud processing method according to claim 1, wherein obtaining point cloud data collected by a ranging device and point cloud density distribution data of the ranging device, comprising:

acquiring at least one frame of scanning pattern corresponding to the scanning mode of the ranging device, wherein the scanning pattern is composed of the mapping points of the point cloud data of the ranging device on the reference plane;

According to the number of the mapping points on the scanning pattern in different statistical regions, the significance coefficient of the point cloud corresponding to the mapping points in each statistical region is determined.
The point cloud processing method according to claim 2, wherein, according to the number of mapping points on the scanning pattern in different statistical regions, determining the significance coefficient corresponding to each statistical region, comprising:

The number of mapping points in a plurality of the statistical regions is normalized to obtain the significance coefficient of the point cloud corresponding to the mapping points in each statistical region.
The point cloud processing method according to claim 2, wherein the size of the statistical area is determined based on the size of the target object predetermined to be detected by the ranging device, wherein the target object includes the first target object and the The second object, the size of the first object is larger than the size of the second object, then the statistical area of the point cloud density distribution data corresponding to the first object has a first size, which is the same as the first object. The statistical region of the point cloud density distribution data corresponding to the two objects has a second size, and the first size is larger than the second size.
The point cloud processing method according to claim 2, wherein the scanning pattern is obtained by mapping the point cloud data output by the ranging device to the reference surface, or the scanning pattern is obtained by mapping The scanning mode of the distance measuring device is obtained by fitting a function.
The point cloud processing method according to claim 1, wherein the reference surface has a plurality of statistical regions, and the point cloud density distribution data includes at least two types of point cloud density distribution data, wherein different types of The point cloud density distribution data has different statistical area sizes.
The point cloud processing method according to claim 6, wherein, according to the scanning mode of the ranging device, acquiring point cloud density distribution data corresponding to the point cloud data collected by the ranging device, further comprising:

The point cloud density distribution data is determined according to the scanning manner and the relationship between the size of the target object predetermined to be identified from the point cloud data and the size of the statistical area.
The point cloud processing method according to claim 7, wherein the point cloud density distribution data is the point cloud density distribution data whose size of the statistical area is smaller than or equal to the size of the target object.
The point cloud processing method according to claim 1, wherein the predetermined value of the point cloud data is determined according to the saliency coefficients corresponding to the mapped points of the point cloud data on the reference plane. processing, including:

According to the depth information represented by the point cloud in the point cloud data and the corresponding significance coefficient, determine the spatial distribution attribute of the point cloud in the point cloud data;

According to the spatial distribution attribute, a predetermined processing manner of the point cloud data is determined.
The point cloud processing method according to claim 9, wherein determining the predetermined processing method of the point cloud data according to the spatial distribution attribute, comprising:

When the spatial distribution attribute corresponding to the first part of the point cloud in the point cloud data is within a first threshold range, determining that the predetermined processing mode of the first part of the point cloud is the first processing mode;

When the spatial distribution attribute corresponding to the second partial point cloud in the point cloud data is within a second threshold range, it is determined that the predetermined processing method of the second partial point cloud is the second processing method.
The point cloud processing method according to claim 10, wherein the first processing method comprises one of the following processing methods: interpolation, upsampling, and time accumulation;

The second processing manner includes downsampling.
The point cloud processing method according to claim 1, wherein the point cloud density distribution data is presented in the form of a heat map, wherein different pixel values in the heat map are used to represent different significance coefficients.
The point cloud processing method according to claim 12, wherein the heat map comprises a first pixel point and a second pixel point, and the pixel value of the first pixel point is greater than the pixel value of the second pixel point , then the significance coefficient corresponding to the first pixel point is greater than the significance coefficient corresponding to the second pixel point; or,

The heat map includes a first pixel point and a second pixel point. The pixel value of the first pixel point is greater than the pixel value of the second pixel point, and the significance coefficient corresponding to the first pixel point is smaller than that of the first pixel point. The significance coefficient corresponding to the second pixel point.
The point cloud processing method according to claim 13, wherein the pixel value comprises a gray value, a color value or a brightness value.
The point cloud processing method according to any one of claims 1 to 14, wherein the reference plane is a plane perpendicular to the central axis of the light pulse sequence emitted by the ranging device.
A point cloud processing device, characterized in that the point cloud processing device comprises:

memory for storing executable instructions;

a processor, configured to execute the instructions stored in the memory, so that the processor performs the following steps:

Acquiring point cloud data collected by the ranging device and point cloud density distribution data of the ranging device, where the point cloud density distribution data is related to the scanning mode of the ranging device, wherein the point cloud density distribution data Including a plurality of significance coefficients, the plurality of significance coefficients are used to characterize the distribution characteristics of the mapping points of the point cloud data on the reference plane;

According to the saliency coefficients corresponding to the mapping points of the point cloud data on the reference plane, a predetermined processing method of the point cloud data is determined, wherein the predetermined processing method includes a first processing method and a second processing method At least one of the processing methods, the first processing method is used to increase the point cloud density in at least part of the area of the point cloud data, and the second processing method is used to reduce the density of the point cloud in at least part of the area in the point cloud data the point cloud density;

According to the predetermined processing manner, the point cloud data is processed.
The point cloud processing device according to claim 16, wherein, according to the scanning mode of the ranging device, acquiring point cloud density distribution data corresponding to the point cloud data collected by the ranging device, comprising:

acquiring at least one frame of scanning pattern corresponding to the scanning mode of the ranging device, wherein the scanning pattern is composed of the mapping points of the point cloud data of the ranging device on the reference plane;

According to the number of the mapping points on the scanning pattern in different statistical regions, the significance coefficient of the point cloud corresponding to the mapping points in each statistical region is determined.
The point cloud processing device according to claim 17, wherein, according to the number of mapping points on the scanning pattern in different statistical regions, determining the significance coefficient corresponding to each statistical region, comprising:

The number of mapping points in a plurality of the statistical regions is normalized to obtain the significance coefficient of the point cloud corresponding to the mapping points in each statistical region.
18. The point cloud processing device according to claim 17, wherein the size of the statistical area is determined based on the size of the target object to be detected by the ranging device, wherein the target object includes the first target object and the The second object, the size of the first object is larger than the size of the second object, then the statistical area of the point cloud density distribution data corresponding to the first object has a first size, which is the same as the first object. The statistical region of the point cloud density distribution data corresponding to the two objects has a second size, and the first size is larger than the second size.
The point cloud processing device according to claim 17, wherein the scanning pattern is obtained by mapping the point cloud data output by the ranging device to the reference surface, or the scanning pattern is obtained by mapping The scanning mode of the distance measuring device is obtained by fitting a function.
The point cloud processing apparatus according to claim 16, wherein the reference surface has a plurality of statistical regions, and the point cloud density distribution data includes at least two types of point cloud density distribution data, wherein different types of The point cloud density distribution data has different statistical area sizes.
The point cloud processing device according to claim 21, wherein, according to the scanning mode of the ranging device, acquiring point cloud density distribution data corresponding to the point cloud data collected by the ranging device, further comprising:

The point cloud density distribution data is determined according to the scanning manner and the relationship between the size of the target object predetermined to be identified from the point cloud data and the size of the statistical area.
The point cloud processing device according to claim 22, wherein the point cloud density distribution data is point cloud density distribution data whose size of the statistical area is smaller than or equal to the size of the target object.
The point cloud processing apparatus according to claim 16, wherein the predetermined value of the point cloud data is determined according to the saliency coefficients corresponding to the mapped points of the point cloud data on the reference plane processing, including:

According to the depth information represented by the point cloud in the point cloud data and the corresponding significance coefficient, determine the spatial distribution attribute of the point cloud in the point cloud data;

According to the spatial distribution attribute, a predetermined processing manner of the point cloud data is determined.
The point cloud processing device according to claim 24, wherein determining a predetermined processing method for the point cloud data according to the spatial distribution attribute, comprising:

When the spatial distribution attribute corresponding to the first part of the point cloud in the point cloud data is within a first threshold range, determining that the predetermined processing mode of the first part of the point cloud is the first processing mode;

When the spatial distribution attribute corresponding to the second partial point cloud in the point cloud data is within a second threshold range, it is determined that the predetermined processing method of the second partial point cloud is the second processing method.
The point cloud processing device according to claim 25, wherein the first processing method comprises one of the following processing methods: interpolation, upsampling, and time accumulation;

The second processing manner includes downsampling.
The point cloud processing device according to claim 16, wherein the point cloud density distribution data is presented in the form of a heat map, wherein different pixel values in the heat map are used to represent different significance coefficients.
The point cloud processing apparatus according to claim 27, wherein the heat map includes a first pixel point and a second pixel point, and the pixel value of the first pixel point is greater than the pixel value of the second pixel point , then the significance coefficient corresponding to the first pixel point is greater than the significance coefficient corresponding to the second pixel point; or,

The heat map includes a first pixel point and a second pixel point. The pixel value of the first pixel point is greater than the pixel value of the second pixel point, and the significance coefficient corresponding to the first pixel point is smaller than that of the first pixel point. The significance coefficient corresponding to the second pixel point.
The point cloud processing apparatus according to claim 28, wherein the pixel value comprises a gray value, a color value or a brightness value.
The point cloud processing device according to any one of claims 16 to 29, wherein the reference plane is a plane perpendicular to the central axis of the light pulse sequence emitted by the ranging device.
A movable platform, characterized in that the movable platform comprises:

Movable platform body;

at least one ranging device, arranged on the movable platform body, for collecting point cloud data of the target scene;

The point cloud processing device according to any one of claims 16 to 30.
The movable platform of claim 31 , wherein the movable platform comprises an aircraft, a robot, a vehicle, a gimbal, or a boat.
A computer storage medium on which a computer program is stored, characterized in that, when the computer program is executed by a processor, the point cloud processing method described in any one of 1 to 15 is implemented.