CN114026410A - Point cloud coloring method, point cloud coloring system, and computer storage medium - Google Patents
Point cloud coloring method, point cloud coloring system, and computer storage medium Download PDFInfo
- Publication number
- CN114026410A CN114026410A CN202080005200.XA CN202080005200A CN114026410A CN 114026410 A CN114026410 A CN 114026410A CN 202080005200 A CN202080005200 A CN 202080005200A CN 114026410 A CN114026410 A CN 114026410A
- Authority
- CN
- China
- Prior art keywords
- reflectivity
- point cloud
- point
- ground
- sub
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 68
- 238000004040 coloring Methods 0.000 title claims abstract description 42
- 238000003860 storage Methods 0.000 title claims abstract description 19
- 238000002310 reflectometry Methods 0.000 claims abstract description 193
- 230000015654 memory Effects 0.000 claims description 22
- 238000004590 computer program Methods 0.000 claims description 8
- 238000012937 correction Methods 0.000 claims description 7
- 238000001514 detection method Methods 0.000 abstract description 10
- 230000011218 segmentation Effects 0.000 abstract description 10
- 230000009286 beneficial effect Effects 0.000 abstract description 2
- 230000003287 optical effect Effects 0.000 description 72
- 238000005070 sampling Methods 0.000 description 13
- 238000004364 calculation method Methods 0.000 description 9
- 230000008859 change Effects 0.000 description 8
- 238000004891 communication Methods 0.000 description 8
- 238000001208 nuclear magnetic resonance pulse sequence Methods 0.000 description 8
- 230000008569 process Effects 0.000 description 7
- 238000010586 diagram Methods 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 5
- 230000006870 function Effects 0.000 description 5
- 230000005540 biological transmission Effects 0.000 description 4
- 239000000463 material Substances 0.000 description 4
- 238000012545 processing Methods 0.000 description 4
- 230000000007 visual effect Effects 0.000 description 4
- 238000004422 calculation algorithm Methods 0.000 description 3
- 239000003086 colorant Substances 0.000 description 3
- 238000009826 distribution Methods 0.000 description 3
- 238000013461 design Methods 0.000 description 2
- 238000001914 filtration Methods 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 239000003550 marker Substances 0.000 description 2
- 238000005259 measurement Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000010606 normalization Methods 0.000 description 2
- 230000010363 phase shift Effects 0.000 description 2
- 230000000630 rising effect Effects 0.000 description 2
- 239000000523 sample Substances 0.000 description 2
- 241000283070 Equus zebra Species 0.000 description 1
- 230000002411 adverse Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 239000011248 coating agent Substances 0.000 description 1
- 238000000576 coating method Methods 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 239000006260 foam Substances 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000004806 packaging method and process Methods 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
- 238000012800 visualization Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
Landscapes
- Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Chemical & Material Sciences (AREA)
- Analytical Chemistry (AREA)
- Biochemistry (AREA)
- General Health & Medical Sciences (AREA)
- General Physics & Mathematics (AREA)
- Immunology (AREA)
- Pathology (AREA)
- Optical Radar Systems And Details Thereof (AREA)
Abstract
A point cloud colorization method, a point cloud colorization system, and a computer storage medium, the point cloud colorization method comprising: acquiring point cloud data, wherein the point cloud data comprises position information and reflectivity information of a point cloud; dividing the point cloud into at least two point clouds of position intervals according to the position information; respectively adjusting the reflectivity of the point clouds in each position interval to reduce the difference of the reflectivity of at least part of the point clouds in the same position interval; and coloring the point cloud according to the adjusted reflectivity. The point cloud coloring method, the point cloud coloring system and the computer storage medium can enable the colored point cloud to be closer to an actual scene, and are beneficial to subsequent detection, segmentation and other applications.
Description
The invention relates to the technical field of laser ranging, in particular to a point cloud coloring method, a point cloud coloring system and a computer storage medium.
Laser radar (LiDAR) senses distance information, position information, reflectivity information, and the like of surrounding objects by actively emitting a laser pulse signal and obtaining a pulse signal reflected by a measured object. The reflectivity information can provide important information about the surface of the measured object, so that algorithms such as segmentation, clustering and visualization based on point cloud are optimized, and the point cloud is generally colored by adopting the reflectivity when the three-dimensional point cloud obtained by scanning is subjected to space display.
Most of the existing coloring schemes of laser radar products perform coloring based on calculated reflectivity or according to echo width, and the higher the reflectivity of the surface of an object in space is or the wider the echo width is, the darker the color of the point is when coloring is performed. However, in practical applications, such a way of coloring point clouds faces many practical problems. Firstly, the same surface of the same object in an actual application scene is affected by surface dirt or unevenness to cause different calculated reflectivity; secondly, in the process of calculating the reflectivity, because only two factors of the pulse width and the distance of the return light are considered, and the factors such as the actual incident angle of the laser and the like are not considered, the reflectivity is not accurately calculated. Therefore, the scheme of coloring only by using the reflectivity has a great influence on the subsequent visual appearance and the subsequent operations of detecting and dividing.
Disclosure of Invention
In this summary, concepts in a simplified form are introduced that are further described in the detailed description. This summary of the invention is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter.
In view of the deficiencies of the prior art, a first aspect of the embodiments of the present invention provides a point cloud coloring method, which includes:
acquiring point cloud data, wherein the point cloud data comprises position information and reflectivity information of a point cloud;
dividing the point cloud into at least two point clouds of position intervals according to the position information;
respectively adjusting the reflectivity of the point clouds in each position interval to reduce the difference of the reflectivity of at least part of the point clouds in the same position interval;
and coloring the point cloud according to the adjusted reflectivity.
A second aspect of an embodiment of the present invention provides a point cloud coloring system, including:
a memory for storing executable instructions;
a processor for executing the instructions stored in the memory, causing the processor to perform the steps of:
acquiring point cloud data, wherein the point cloud data comprises position information and reflectivity information of a point cloud;
dividing the point cloud into at least two point clouds of position intervals according to the position information;
respectively adjusting the reflectivity of the point clouds in each position interval to reduce the difference of the reflectivity of at least part of the point clouds in the same position interval;
and coloring the point cloud according to the adjusted reflectivity.
A third aspect of the embodiments of the present invention provides a computer storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the point cloud coloring method provided by the first aspect of the embodiments of the present invention.
The point cloud coloring method, the point cloud coloring system and the computer storage medium in the embodiment of the invention can enable the point cloud to be more uniformly colored in the display process and to be closer to an actual scene.
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive labor.
FIG. 1 is a block diagram of a ranging device according to an embodiment of the invention;
FIG. 2 is a schematic structural diagram of a distance measuring device according to an embodiment of the present invention;
FIG. 3 shows a schematic flow diagram of a point cloud colorization method in accordance with one embodiment of the present invention;
FIG. 4 shows a block diagram of a point cloud coloring system, according to one embodiment of the invention.
In order to make the objects, technical solutions and advantages of the present invention more apparent, exemplary embodiments according to the present invention will be described in detail below with reference to the accompanying drawings. It is to be understood that the described embodiments are merely a subset of embodiments of the invention and not all embodiments of the invention, with the understanding that the invention is not limited to the example embodiments described herein. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the invention described herein without inventive step, shall fall within the scope of protection of the invention.
In the following description, numerous specific details are set forth in order to provide a more thorough understanding of the present invention. It will be apparent, however, to one skilled in the art, that the present invention may be practiced without one or more of these specific details. In other instances, well-known features have not been described in order to avoid obscuring the invention.
It is to be understood that the present invention may be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. As used herein, the term "and/or" includes any and all combinations of the associated listed items.
In order to provide a thorough understanding of the present invention, a detailed structure will be set forth in the following description in order to explain the present invention. Alternative embodiments of the invention are described in detail below, however, the invention may be practiced in other embodiments that depart from these specific details.
First, a detailed exemplary description will be given of a structure of a distance measuring apparatus according to an embodiment of the present invention with reference to fig. 1 and 2. The distance measuring device may comprise a lidar, which is only an example and other suitable distance measuring devices may be applied to the present application.
The ranging device is used for sensing external environment information, such as distance information, azimuth information, reflection intensity information, speed information, and the like of an environmental target. In one implementation, the distance measuring device may detect the distance of the measured object from the distance measuring device by measuring the Time of Flight (TOF), which is the Time-of-Flight Time, of light propagation between the distance measuring device and the measured object. Alternatively, the distance measuring device may detect the distance from the measured object to the distance measuring device by other techniques, such as a distance measuring method based on phase shift (phase shift) measurement, a distance measuring method based on frequency shift (frequency shift) measurement, and the like, which is not limited herein.
For ease of understanding, the following describes an example of the ranging operation with reference to the ranging apparatus 100 shown in fig. 1.
As an example, the ranging apparatus 100 comprises a transmitting module for transmitting a sequence of light pulses to detect a target scene; the scanning module is used for sequentially changing the propagation paths of the optical pulse sequences transmitted by the transmitting module to different directions for emission to form a scanning view field; the detection module is used for receiving the light pulse sequence reflected back by the object and determining the distance and/or the direction of the object relative to the distance measuring device according to the reflected light pulse sequence so as to generate the point cloud point.
Specifically, as shown in fig. 1, the transmitting module includes a transmitting circuit 110; the detection module includes a receiving circuit 120, a sampling circuit 130, and an arithmetic circuit 140.
The transmit circuit 110 may emit a train of light pulses (e.g., a train of laser pulses). The receiving circuit 120 may receive the optical pulse train reflected by the detected object, that is, obtain the pulse waveform of the echo signal through the optical pulse train, perform photoelectric conversion on the optical pulse train to obtain an electrical signal, process the electrical signal, and output the electrical signal to the sampling circuit 130. The sampling circuit 130 may sample the electrical signal to obtain a sampling result. The arithmetic circuit 140 may determine the distance, i.e., the depth, between the ranging apparatus 100 and the detected object based on the sampling result of the sampling circuit 130.
Optionally, the distance measuring apparatus 100 may further include a control circuit 150, and the control circuit 150 may implement control of other circuits, for example, may control an operating time of each circuit and/or perform parameter setting on each circuit, and the like.
It should be understood that, although the distance measuring device shown in fig. 1 includes a transmitting circuit, a receiving circuit, a sampling circuit and an arithmetic circuit for emitting a light beam to detect, the embodiments of the present application are not limited thereto, and the number of any one of the transmitting circuit, the receiving circuit, the sampling circuit and the arithmetic circuit may be at least two, and the at least two light beams are emitted in the same direction or in different directions respectively; the at least two light paths may be emitted simultaneously or at different times. In one example, the light emitting chips in the at least two transmitting circuits are packaged in the same module. For example, each transmitting circuit comprises a laser emitting chip, and die of the laser emitting chips in the at least two transmitting circuits are packaged together and accommodated in the same packaging space.
In some implementations, in addition to the circuit shown in fig. 1, the distance measuring apparatus 100 may further include a scanning module, configured to change a propagation direction of at least one light pulse sequence (e.g., a laser pulse sequence) emitted by the emitting circuit to emit light, so as to scan the field of view. Illustratively, the scan area of the scan module within the field of view of the ranging device increases over time.
Here, a module including the transmission circuit 110, the reception circuit 120, the sampling circuit 130, and the operation circuit 140, or a module including the transmission circuit 110, the reception circuit 120, the sampling circuit 130, the operation circuit 140, and the control circuit 150 may be referred to as a ranging module, which may be independent of other modules, for example, a scanning module.
The distance measuring device can adopt a coaxial light path, namely the light beam emitted by the distance measuring device and the reflected light beam share at least part of the light path in the distance measuring device. For example, at least one path of laser pulse sequence emitted by the emitting circuit is emitted by the scanning module after the propagation direction is changed, and the laser pulse sequence reflected by the detector is emitted to the receiving circuit after passing through the scanning module. Alternatively, the distance measuring device may also adopt an off-axis optical path, that is, the light beam emitted by the distance measuring device and the reflected light beam are transmitted along different optical paths in the distance measuring device. FIG. 2 is a schematic diagram of one embodiment of the distance measuring device of the present invention using coaxial optical paths.
The ranging apparatus 200 comprises a ranging module 210, the ranging module 210 comprising an emitter 203 (which may comprise the transmitting circuitry described above), a collimating element 204, a detector 205 (which may comprise the receiving circuitry, sampling circuitry and arithmetic circuitry described above) and a path-altering element 206. The distance measuring module 210 is configured to emit a light beam, receive return light, and convert the return light into an electrical signal. Wherein the emitter 203 may be configured to emit a sequence of light pulses. In one embodiment, the transmitter 203 may emit a sequence of laser pulses. Optionally, the laser beam emitted by the emitter 203 is a narrow bandwidth beam having a wavelength outside the visible range. The collimating element 204 is disposed on an emitting light path of the emitter, and is configured to collimate the light beam emitted from the emitter 203, and collimate the light beam emitted from the emitter 203 into parallel light to be emitted to the scanning module. The collimating element is also for converging at least a portion of the return light reflected by the detector. The collimating element 204 may be a collimating lens or other element capable of collimating a light beam.
In the embodiment shown in fig. 2, the transmit and receive optical paths within the distance measuring device are combined by the optical path altering element 206 before the collimating element 204, so that the transmit and receive optical paths may share the same collimating element, making the optical path more compact. In other implementations, the emitter 203 and the detector 205 may use respective collimating elements, and the optical path changing element 206 may be disposed in the optical path after the collimating elements.
In the embodiment shown in fig. 2, since the beam aperture of the light beam emitted from the emitter 203 is small and the beam aperture of the return light received by the distance measuring device is large, the optical path changing element can adopt a small-area mirror to combine the emission optical path and the reception optical path. In other implementations, the optical path changing element may also be a mirror with a through hole, wherein the through hole is used for transmitting the outgoing light from the emitter 203, and the mirror is used for reflecting the return light to the detector 205. Therefore, the shielding of the bracket of the small reflector to the return light can be reduced in the case of adopting the small reflector.
In the embodiment shown in fig. 2, the optical path altering element is offset from the optical axis of the collimating element 204. In other implementations, the optical path altering element may also be located on the optical axis of the collimating element 204.
The ranging device 200 also includes a scanning module 202. The scanning module 202 is disposed on the emitting light path of the distance measuring module 210, and the scanning module 202 is configured to change the transmission direction of the collimated light beam 219 emitted by the collimating element 204, project the collimated light beam to the external environment, and project the return light beam to the collimating element 204. The return light is converged by the collimating element 204 onto the detector 205.
In one embodiment, the scanning module 202 may include at least one optical element for changing the propagation path of the light beam, wherein the optical element may change the propagation path of the light beam by reflecting, refracting, diffracting, etc. the optical element includes at least one light refracting element having non-parallel exit and entrance faces, for example. For example, the scanning module 202 includes a lens, mirror, prism, galvanometer, grating, liquid crystal, Optical Phased Array (Optical Phased Array), or any combination thereof. In one example, at least a portion of the optical element is moved, for example, by a driving module, and the moved optical element can reflect, refract, or diffract the light beam to different directions at different times. In some embodiments, multiple optical elements of the scanning module 202 may rotate or oscillate about a common axis 209, with each rotating or oscillating optical element serving to constantly change the direction of propagation of an incident beam. In one embodiment, the multiple optical elements of the scanning module 202 may rotate at different rotational speeds or oscillate at different speeds. In another embodiment, at least some of the optical elements of the scanning module 202 may rotate at substantially the same rotational speed. In some embodiments, the multiple optical elements of the scanning module may also be rotated about different axes. In some embodiments, the multiple optical elements of the scanning module may also rotate in the same direction, or in different directions; or in the same direction, or in different directions, without limitation.
In one embodiment, the scanning module 202 includes a first optical element 214 and a driver 216 coupled to the first optical element 214, the driver 216 configured to drive the first optical element 214 to rotate about the rotation axis 209, such that the first optical element 214 redirects the collimated light beam 219. The first optical element 214 projects the collimated beam 219 into different directions. In one embodiment, the angle between the direction of the collimated beam 219 after it is altered by the first optical element and the axis of rotation 209 changes as the first optical element 214 is rotated. In one embodiment, the first optical element 214 includes a pair of opposing non-parallel surfaces through which the collimated light beam 219 passes. In one embodiment, the first optical element 214 includes a prism having a thickness that varies along at least one radial direction. In one embodiment, the first optical element 214 comprises a wedge angle prism that refracts the collimated beam 219.
In one embodiment, the scanning module 202 further comprises a second optical element 215, the second optical element 215 rotating around a rotation axis 209, the rotation speed of the second optical element 215 being different from the rotation speed of the first optical element 214. The second optical element 215 is used to change the direction of the light beam projected by the first optical element 214. In one embodiment, the second optical element 215 is coupled to another driver 217, and the driver 217 drives the second optical element 215 to rotate. The first optical element 214 and the second optical element 215 may be driven by the same or different drivers, such that the first optical element 214 and the second optical element 215 rotate at different speeds and/or turns, thereby projecting the collimated light beam 219 into different directions in the ambient space, which may scan a larger spatial range. In one embodiment, the controller 218 controls the drivers 216 and 217 to drive the first optical element 214 and the second optical element 215, respectively. The rotation speed of the first optical element 214 and the second optical element 215 can be determined according to the region and the pattern expected to be scanned in the actual application. The drives 216 and 217 may include motors or other drives.
In one embodiment, second optical element 215 includes a pair of opposing non-parallel surfaces through which the light beam passes. In one embodiment, second optical element 215 includes a prism having a thickness that varies along at least one radial direction. In one embodiment, second optical element 215 comprises a wedge angle prism.
In one embodiment, the scan module 202 further comprises a third optical element (not shown) and a driver for driving the third optical element to move. Optionally, the third optical element comprises a pair of opposed non-parallel surfaces through which the light beam passes. In one embodiment, the third optical element comprises a prism having a thickness that varies along at least one radial direction. In one embodiment, the third optical element comprises a wedge angle prism. At least two of the first, second and third optical elements rotate at different rotational speeds and/or rotational directions.
In one embodiment, the scanning module comprises 2 or 3 photorefractive elements arranged in sequence on an outgoing light path of the optical pulse sequence. Optionally, at least 2 of the photorefractive elements in the scanning module rotate during scanning to change the direction of the sequence of light pulses.
The scanning module has different scanning paths at least partially different times, and the rotation of each optical element in the scanning module 202 may project light in different directions, such as the direction of the projected light 211 and the direction 213, so as to scan the space around the distance measuring device 200. When the light 211 projected by the scanning module 202 hits the detection object 201, a part of the light is reflected by the detection object 201 to the distance measuring device 200 in the opposite direction to the projected light 211. The return light 212 reflected by the object 201 passes through the scanning module 202 and then enters the collimating element 204.
The detector 205 is placed on the same side of the collimating element 204 as the emitter 203, and the detector 205 is used to convert at least part of the return light passing through the collimating element 204 into an electrical signal.
In one embodiment, each optical element is coated with an antireflection coating. Optionally, the thickness of the antireflection film is equal to or close to the wavelength of the light beam emitted by the emitter 203, which can increase the intensity of the transmitted light beam.
In one embodiment, a filter layer is coated on a surface of a component in the distance measuring device, which is located on the light beam propagation path, or a filter is arranged on the light beam propagation path, and is used for transmitting at least a wave band in which the light beam emitted by the emitter is located and reflecting other wave bands, so as to reduce noise brought to the receiver by ambient light.
In some embodiments, the transmitter 203 may include a laser diode through which laser pulses in the order of nanoseconds are emitted. Further, the laser pulse reception time may be determined, for example, by detecting the rising edge time and/or the falling edge time of the electrical signal pulse. In this manner, the ranging apparatus 200 may calculate TOF using the pulse reception time information and the pulse emission time information, thereby determining the distance of the probe 201 to the ranging apparatus 200. The distance and orientation detected by ranging device 200 may be used for remote sensing, obstacle avoidance, mapping, modeling, navigation, and the like.
Existing distance measuring devices typically color the point cloud based on reflectivity when displaying the point cloud. For the calculation of the reflectivity, the return light energy value is usually calculated according to the relationship between the pulse width and the return light energy calibrated in advance, and the reflectivity is obtained by multiplying the energy value by the square of the distance. However, in fact, the return light energy is not only related to the distance and the reflectivity, but also affected by the incident angle, the laser spot size, and the like, which cannot be directly obtained, and thus some random errors are introduced in the calculation of the reflectivity. Directly utilizing the reflectivity for point cloud coloring can introduce noise, resulting in poor visual effect and adverse effects on subsequent point cloud application.
In view of the above problems, the present invention provides a point cloud coloring method, and the point cloud coloring method of the present invention is first described with reference to fig. 3. As shown in fig. 3, the point cloud coloring method 300 includes the following steps:
in step S310, point cloud data is acquired, wherein the point cloud data includes position information and reflectivity information of a point cloud;
in step S320, dividing the point cloud into at least two point clouds of location intervals according to the location information;
in step S330, respectively adjusting the reflectivity of the point clouds in each location interval to reduce the difference of the reflectivity of at least part of the point clouds in the same location interval;
in step S340, the point cloud is colored according to the adjusted reflectivity.
The point cloud coloring method 300 of the embodiment of the invention can reduce the difference of the reflectivity of at least part of point clouds in the same position interval, avoid the problem of different measured reflectivities of the same object caused by different incident angles and the like, ensure that the point clouds are colored more uniformly in the display process, are closer to the actual scene, improve the visual effect, and are beneficial to subsequent detection, segmentation and other applications.
For example, in step S310, a laser pulse may be actively emitted to the measured object through the ranging device, a laser echo signal may be captured, and information of the measured object may be obtained according to the echo signal, for example, a distance of the measured object may be calculated according to a time difference between emission and reception of the laser, and angle information of the measured object may be obtained based on a known emission direction of the laser. Through high-frequency transmission and reception, spatial position information such as distance and angle information of a large number of detection points can be acquired, and the detection points are called point clouds.
In one embodiment, the original point cloud data collected by the ranging apparatus obtained in step S310 includes pulse width information (i.e., a difference between times of a rising edge and a falling edge of a pulse signal reaching a certain voltage value) and distance information, and the reflectivity information in the point cloud data can be calculated according to the pulse width information and the distance information.
Specifically, the pulse width information is used to calculate the return light energy of the reflected signal, and the return light energy is related to the distance, reflectivity, incident angle, laser spot size, and the like of the measured object. Because the incident angle and the laser spot size cannot be directly obtained, the return light energy received by the laser radar is only considered to be influenced by two factors of the distance and the reflectivity, i.e. the return light energy is considered to be in direct proportion to the reflectivity and in inverse proportion to the square of the distance, as shown in formula (1):
E intensity＝k*r/R 2 (1)
wherein E isintensityIs the energy value, R is the reflectivity, R is the distance between the lidar and the object to be measured, and k is the coefficient associated with the lidar.
For the calculation of the return light energy, the data of the laser radar at normal incidence at different distances can be collected in advance by fixed points, and the relationship between the pulse width and the return light energy is calibrated. After the pulse width information in the point cloud data is acquired, the return light energy can be calculated according to the relationship calibrated in advance. In addition to the pulse width, the relationship between the echo energy and the characteristics such as the echo height and the echo area may be calibrated in advance, and the echo energy may be calculated from the relationship between the characteristics and the echo energy calibrated in advance.
By way of example, in calibrating the pulse width to energy relationship, only normal incidence is considered, and oblique incidence is not considered. During calibration, the rotation speed of the laser radar may be set to 0, a single point may be aimed at a target, light pulse signals may be transmitted by the laser radar to a plurality of targets with known true reflectivity values at different distances (e.g., 0.5m intervals within 10m, 1m intervals within 20m, and 2m intervals outside 20 m), reflected pulse signals reflected by the target may be received, and the reflected pulse signals may be sampled to obtain sampling results. And then, determining the pulse width of the reflected pulse signal based on the sampling result, obtaining the return light energy based on the ratio of the reflectivity true value of each target object to the square of the distance true value of each target object distance measuring device, and fitting a relation curve based on the pulse width and the return light energy to obtain the relation between the pulse width and the return light energy. The target can be black and white foam or a target with a known reflectivity true value made of other materials. Illustratively, in order to improve the accuracy of calibration and increase the calculation speed, a cubic polynomial piecewise fitting is adopted under each of the above conditions.
Illustratively, before each reflectivity calculation, it is first determined whether the pulse width is stretched, and the stretched waveform needs to be corrected for application to the pulse width-echo energy curve. For the stretched waveform, it is possible to correct it with the difference between the actually measured pulse width and the pulse width corrected with the height information as a parameter, determine the final output pulse width based on the measured pulse width and the corrected pulse width, and limit the output pulse width within a range.
According to the pulse width selection and correction model and the relationship between the pulse width and the return light energy obtained by fitting, the return light energy values of the positions of different points in the point cloud can be calculated. Then, the calculated return light energy value is multiplied by the square of the distance between the point and the position, so as to obtain the value of the reflectivity of the position where the point is located, as shown in formula (2):
r＝R 2/k*E intensity＝R 2/k*f(pulse_width) (2)
wherein, pulse _ width is the pulse width of the echo. Thus, the reflectance calculation is translated as a function of distance and pulse width.
Further, some errors inevitably exist in the process of material selection, production and equipment of the distance measuring device such as the laser radar, and these errors are finally reflected in the calculation of the depth, angle and reflectivity of the point cloud data, as shown in table 1:
TABLE 1
| Laser radar serial number | Black color | Grey colour | Green colour | White colour |
| 475 | 4.9 | 33.1 | 61.4 | 64.2 |
| 480 | 6.0 | 67.4 | 87.2 | 107.2 |
| 481 | 6.7 | 51.8 | 69 | 75.7 |
| 482 | 6.6 | 72.6 | 100.2 | 117.2 |
Table 1 shows the measured reflectance values of different lidar materials at the same location for different reflectances. As can be seen from table 1, the measured reflectivities of different lidar materials of the same color are different greatly, i.e. the consistency between different devices is poor. Therefore, after the reflectivity is obtained in the above manner, further individual difference consistency correction is still required for the reflectivity of different lidar. During consistency correction, a linear factor correction mode can be used, namely, a linear factor calibrated in advance for the laser radar is used for correcting the reflectivity, so that the reflectivity is generally closer to a true value, and the corrected reflectivity is shown in table 2:
TABLE 2
| Laser radar serial number | Black color | Grey colour | Green colour | White colour |
| 475 | 7.0 | 47.5 | 88.1 | 92.1 |
| 480 | 4.3 | 48.4 | 62.3 | 76.6 |
| 481 | 6.9 | 53.6 | 71.3 | 78.3 |
| 482 | 4.7 | 51.2 | 70.7 | 82.7 |
As can be seen from table 2, after the individual difference consistency correction is performed, the difference between the measured reflectances of different lidar is reduced, and the error from the true reflectivity is reduced.
Next, in step S320, the point cloud is divided into at least two point clouds of position intervals according to the position information of the point cloud in the point cloud data. The target area in the point cloud may be determined first, and the target area may be divided into at least two location intervals. Alternatively, the target area may be not determined, and the position section may be divided for the entire current frame point cloud. The target area may be selected according to application requirements, for example for a traveling vehicle, the target area may be a road area.
The division of the location interval can also be set according to the actual application requirements. In one embodiment, for example, for a road surface scene, the above location intervals may be ground intervals and ground intervals, and the point clouds in the ground intervals are ground points, such as point clouds including vehicles, trees, and the like; the point clouds in the ground interval are ground points.
The ground points and the ground points in the point cloud can be segmented by any suitable method. For example, ground points may be first segmented from the point cloud, and cloud points other than the ground points may be considered to be above-ground points.
As an example, the step of segmenting ground points from the point cloud may comprise: firstly, determining at least part of ground points according to the heights of point cloud points in point cloud data; and then, carrying out plane fitting on the rest point cloud points, and filtering out point cloud points of which the distance between the point cloud points and the plane obtained by fitting is greater than a threshold value. And then, the residual point cloud points are the ground points.
Wherein determining at least some ground points may comprise: firstly, rasterizing a horizontal plane of a point cloud space, and regarding each grid, taking the height of a point cloud point with the lowest height among all point cloud points as a reference height of the grid, thereby obtaining a minimum height distribution map of the point cloud space; and then setting a certain height threshold value based on the minimum height distribution map, and filtering out point cloud points of which the height difference between each grid and the reference height of the corresponding grid is greater than the threshold value. And then, on the basis, removing the point cloud points with the absolute height larger than a certain threshold value according to the absolute heights of all the point cloud points in the point cloud space. And then, the retained point cloud points are the ground points. And after the ground points are determined, point cloud points except the ground points are the ground points.
It should be understood that the embodiment of the present invention is not limited by the specifically adopted point cloud segmentation method, and the existing point cloud segmentation method or the point cloud segmentation method developed in the future can be applied to the point cloud coloring method according to the embodiment of the present invention.
In step S330, the reflectivity of the point clouds in each location interval is adjusted to reduce the difference of the reflectivity of at least some point clouds in the same location interval. By reducing the difference of the reflectivity of at least part of point clouds in the same position interval, the phenomenon of uneven reflectivity of the same object can be reduced.
As described above, when the location section divided in the point cloud includes the above-ground point and the ground point, the adjustment performed in step S330 may include reducing the difference in reflectivity of at least part of the above-ground point and reducing the difference in reflectivity of at least part of the ground point. Specifically, the reducing the difference in the reflectivity of at least a part of the point clouds in the same position interval may include reducing the difference in the reflectivity of the point clouds of the same object in the same position interval.
Since only the normal incidence mode is considered in calculating and fitting the reflectivity, and the angle problem of the incidence plane is not considered, the reflectivity calculation deviation may be caused by the influence of the object surface in practical application, for example, in a road scene, the reflectivity of a vehicle body and a road surface and the reflectivity of trees are disordered, the noise is more, the visual impression is influenced, and the subsequent tasks such as detection, identification, segmentation and the like based on the reflectivity characteristics are influenced. According to the embodiment of the invention, after the reflectivity of the point clouds in each position interval is adjusted, the difference of the reflectivity of at least part of the point clouds in the same position interval is reduced, for example, when the target area is a road area, the difference between ground points or the difference between ground points is reduced by the adjustment, so that the problem that the point cloud reflectivity of the same object is too large due to different incidence angles is reduced, and the reflectivity of the processed point clouds is more uniform.
Further, adjusting the reflectivity of the point cloud may also include increasing a difference in reflectivity of the point cloud between different location intervals. When the location interval segmented in the point cloud includes the above-ground point and the ground point, the adjusting of the reflectance of the point cloud includes increasing a difference between the reflectances of the above-ground point and the ground point. As an example, reducing the difference in the reflectivity of the point clouds in the same location section and increasing the difference in the reflectivity of the point clouds between different location sections includes, but is not limited to, normalizing the reflectivity of the point clouds in different location sections according to different trends. By increasing the difference of the reflectivity between different position intervals, the point clouds in different position intervals can be more obviously distinguished when the point clouds are colored by the reflectivity subsequently.
Furthermore, each position interval can be divided into a plurality of sub-position intervals, and the reflectivity is further adjusted by taking the sub-position intervals as a unit, so that the difference of the reflectivity of the point clouds of the same object is reduced, and the difference of the reflectivity of the point clouds of different objects is enlarged.
For example, the ground points may be divided into a plurality of first sub-location sections, and the reflectivity of the ground points in each first sub-location section may be adjusted. As an example, the first sub-location interval may be segmented according to a distance between the point cloud and a ranging device that generated the point cloud.
Specifically, the ground point may be divided into a plurality of two-dimensional mesh ground points, and each first sub-location section includes at least one mesh, that is, each mesh may form one first sub-location section, or a plurality of meshes may form one first sub-location section together. Since the real ground is not a regular plane, it is possible to map ground points onto a horizontal plane and divide the horizontal plane into a plurality of two-dimensional meshes, the ground points mapped into each two-dimensional mesh being divided into ground points in the corresponding mesh. The meshes divided in the horizontal plane may have the same shape and size.
In a road scene, the ground points generally include ground points of a common road surface and ground points of road surface marks such as lane lines and zebra stripes. After the ground point is divided into a plurality of first sub-position intervals, the point cloud in each first sub-position interval may include only the ordinary road surface points, only the road surface identification points, or both the ordinary road surface points and the road surface identification points. The embodiment of the invention adjusts the reflectivity of the ground points of the first sub-position intervals to reduce the difference between the reflectivity of the common road surface and the reflectivity of the interior of the road surface marking point and make the difference between the common road surface and the road surface marking point more obvious.
In order to achieve an increase in the difference between the reflectivities of the ground points of different surfaces and a decrease in the difference between the reflectivities of the ground points of the same surface, the reflectivities of the ground points in each first sub-location interval may be adjusted as follows:
when the difference between the reflectivities of the ground points in the first sub-location section does not exceed the preset threshold, the first sub-location section is considered to include the ground points of the same surface, such as only the ordinary road points or only the road marking points, and thus the reflectivities of the ground points in the first sub-location section are adjusted in the same trend. For example, the reflectivity of all ground points within the first sub-location interval may be adjusted to trend toward a larger or smaller range of values of reflectivity within the first sub-location interval. At this time, the adjustment direction can be judged by integrating a plurality of first sub-position intervals, and when the reflectivity of the first sub-position interval is judged to be larger, the reflectivity can be adjusted towards a larger value interval; when the reflectivity of the first sub-position interval is judged to be smaller, the reflectivity can be adjusted towards a smaller value interval.
Accordingly, when the difference between the reflectivities of the ground points in the first sub-location-interval exceeds the preset threshold, the ground points in the first sub-location-interval are considered to include different surfaces, such as both the ordinary road surface points and the road surface marking points, so that the reflectivities of the ground points in the first sub-location-interval are adjusted according to different trends to expand the difference of the different surface reflectivities and simultaneously reduce the difference of the same surface reflectivity. Specifically, the reflectivity of the ground points in the first sub-location section may be adjusted toward two value sections where the reflectivity of the ground points in the first sub-location section is located at both ends, respectively.
For example, if the first sub-location section includes both the ground points of the ordinary road surface and the ground points of the road sign, where the reflectivity of the ordinary road surface is small, the reflectivity of the road sign mark line is large, and the difference between the two exceeds a preset threshold, the reflectivities of the ground points with small reflectivity in the first sub-location section (i.e., the ground points of the ordinary road surface) are all adjusted toward the numerical value section with the minimum reflectivity in the first sub-location section, so as to reduce the difference between the reflectivities of the ground points of the ordinary road surface; simultaneously, the reflectivity of the ground points with larger reflectivity (namely the ground points of the road surface marks) in the first sub-position interval is adjusted towards the numerical value interval with the maximum reflectivity in the first sub-position interval in a same direction, so that the difference between the reflectivity of the ground points of the road surface marks is reduced; in addition, after the convergence adjustment is performed to the numerical value sections at both ends, the difference between the ground point of the ordinary road surface and the ground point of the road surface mark is obviously increased.
In some embodiments, the larger value interval of the reflectivity of the ground points in the different first sub-location intervals may be adjusted in a converging manner, and the smaller value interval of the reflectivity of the ground points in the different first sub-location intervals may be adjusted in a converging manner, so as to make the reflectivity of the different first sub-location intervals more uniform. For example, according to the above description, for the ground points of the normal road surface and the ground points of the road surface marker, in each first sub-position section, regardless of whether the first sub-position section includes the same type of ground points or different types of ground points, the numerical value section in which the reflectance of the ground points of the normal road surface is small is adjusted, and the numerical value section in which the reflectance of the ground points of the road surface marker exceeds large is adjusted. Therefore, through the adjustment, the difference between the interior of the common road surface point and the interior of the road surface marking point between different first sub-position intervals can be reduced.
The above-ground point may be divided into at least two second sub-location sections, and the reflectivity of the above-ground point in each second sub-location section may be adjusted.
As an example, since objects of different categories in the above-ground point are generally distributed in different height intervals, the above-ground point may be divided according to height. Specifically, the ground point may be divided into the ground points of the at least two second sub-position sections according to the height of the ground point above the plane where the ground point is located, with the plane where the ground point is located as a reference. For example, when applied to automatic driving, since the heights of vehicles that are more concerned in a road scene are generally distributed within a certain height range above the road surface, and the degree of attention to objects above this height range is low, the ground point within a first height above the plane where the ground point is located may be determined as the ground point of the first and second sub-position sections, and the ground point between the first height and the second height above the plane where the ground point is located may be determined as the ground point of the second sub-position section. For example, the first and second sub-intervals may have a height in the range of 0-2 meters and the second and second sub-intervals may have a height in the range of 2-5 meters. The height interval of the ground point can be reasonably set according to needs, and is not particularly limited herein. Of course, the above-ground point may also include more than two height intervals, which is not limited herein.
The adjusting of the reflectivity of the second sub-location interval may comprise reducing the difference in reflectivity of the upper points within the same second sub-location interval and expanding the difference in reflectivity of the upper points within different second sub-location intervals. For example, the ground points in the first and second sub-location sections generally include point clouds of vehicles, and the ground points in the second and second sub-location sections generally include point clouds of trees on both sides of roads, so that the adjustment can reduce the difference between the point clouds of vehicles and the point clouds of trees and expand the difference between the point clouds of vehicles and the point clouds of trees. In some embodiments, the adjustment of the reflectivity of the second sub-location section may also be similar to the adjustment of the reflectivity of the first sub-location section, for example, a plurality of second sub-location sections may be divided, the point clouds including the same object or the point clouds including different objects in each second sub-location section are respectively determined, and the difference between the point clouds including the same object is reduced, and the difference between the point clouds including different objects is enlarged.
In addition, the incident angle of the emergent light of the distance measuring device on the surface of the measured object is also influenced by the position of the measured object in the field of view, for example, the incident angle at the near position of the road surface is small, the incident angle at the far position is relatively large, the incident angle at the central area close to the field of view is small, and the incident angle at the edge area close to the field of view is large; for the same building, the angle of incidence is smaller at lower positions and larger at higher positions. The different angles of incidence will affect the return light energy and thus the reflectivity measured from the return light energy. Thus, in one embodiment, the point cloud colorization method 300 may further comprise: determining an adjustment coefficient according to position information in the point cloud data, and adjusting the reflectivity of the point cloud by using the adjustment coefficient, wherein the adjustment coefficients of the point cloud at least two different positions are different. Wherein the location information comprises at least one of: distance information, height information, horizontal position information in the current field of view.
Specifically, the adjustment coefficient may vary with the change of the position information, for example, may vary in a gradient or linearly with the change of the distance, height, or horizontal position information. A plurality of adjustment coefficients may be determined from different position information and a plurality of adjustments may be performed. The adjustment of the reflectivity using the adjustment coefficient associated with the distance information may be performed before or after performing the adjustment described in step S330.
Taking the height as an example, the reflectance of the point cloud can be normalized and adjusted according to the following formula (3) at different heights:
where r is the reflectance value solved before normalization, point.z is the height value of the current point, z is the value of the current pointmaxAnd z _ min respectively refer to the maximum value and the minimum value of the point cloud height of the current frame, and R is a reflectivity value after height normalization.
In step S340, the point cloud is colored according to the adjusted reflectivity.
For example, for each point cloud point, if it is detected that the reflectivity of the point belongs to a preset reflectivity interval, the pixel parameter corresponding to the reflectivity interval is assigned to the point cloud point.
The pixel parameter corresponding to each reflectivity interval may include at least one of a pixel color and a pixel gray scale. For example, the pixel colors may be represented by three channel color values, and the three channel color values are different for different pixel colors. And if a certain point cloud point belongs to a certain reflectivity interval, giving a three-channel color value corresponding to the reflectivity interval to the point cloud point, so that the color displayed by the point cloud point is the color corresponding to the three-channel color value. Different measured objects can be displayed in different colors due to different measured objects having different reflectivity.
Before the reflectivity of the point cloud is not adjusted, the reflectivity measured at different positions of the same object is different due to random errors in the calculation process of the point cloud reflectivity and inconsistency in the production process of radar equipment, and errors are generated in the identification and segmentation results due to the influence of the reflectivity when the identification and segmentation are carried out.
According to the point cloud coloring method 300 provided by the embodiment of the invention, the reflectivity distribution becomes uniform in the whole space, so that the reflectivity of the same object in the scene is more consistent, the contrast of different objects is more obvious, and the stability and accuracy of algorithms such as detection and segmentation are better promoted.
In the following, an embodiment of the point cloud coloring system according to the invention is described with reference to fig. 4, wherein the features of the distance measuring device described above can be incorporated into the embodiment. The point cloud coloring system can be realized as a computer, a server or an electronic device such as a vehicle-mounted terminal.
In some embodiments, the point cloud colorization system 400 as described in fig. 4 further comprises one or more processors 410, one or more memories 420, the one or more processors 410 working in conjunction or separately. Optionally, the point cloud colorization system 400 may also include at least one of an input device (not shown), an output device (not shown), and an image sensor (not shown), which are interconnected by a bus system and/or other form of connection mechanism (not shown).
The memory 420 is used for storing program instructions executable by the processor, for example, for storing corresponding steps and program instructions for implementing a point cloud coloring method according to an embodiment of the present invention. May include one or more computer program products that may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory. The volatile memory may include, for example, Random Access Memory (RAM), cache memory (cache), and/or the like. The non-volatile memory may include, for example, Read Only Memory (ROM), hard disk, flash memory, etc.
The input device may be a device used by a user to input instructions and may include one or more of a keyboard, a mouse, a microphone, a touch screen, and the like.
The output device may output various information (e.g., images or sounds) to an outside (e.g., a user), and may include one or more of a display, a speaker, etc. for outputting the colored point cloud as an image or video.
The communication interface (not shown) is used for communication with other devices, including wired or wireless communication. The ranging device may access a wireless network based on a communication standard, such as WiFi, 2G, 3G, 4G, 5G, or a combination thereof. In one exemplary embodiment, the communication interface receives a broadcast signal or broadcast associated information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the communication interface further comprises a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, Ultra Wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.
The point cloud colorization system 400 includes one or more processors, working collectively or individually, a memory for storing program instructions; the processor is configured to execute the program instructions stored in the memory, and when the program instructions are executed, the processor is configured to implement the corresponding steps in the point cloud coloring method according to the embodiment of the invention, and for avoiding repetition, specific descriptions of the methods may refer to the related descriptions of the foregoing embodiments.
In one embodiment, the point cloud coloring system of embodiments of the present invention may be applied to a movable platform. The movable platform with the point cloud coloring system can measure the external environment, for example, the distance between the movable platform and an obstacle is measured for the purpose of avoiding obstacles, and the two-dimensional or three-dimensional mapping is carried out on the external environment. In some embodiments, the movable platform comprises an unmanned vehicle or a vehicle equipped with an Advanced Driving Assistance System (ADAS). The movable platform may further comprise at least one of an unmanned aerial vehicle, a robot, a boat, a camera.
The point cloud coloring system and the mobile platform in the embodiment of the invention have the same advantages as the method because the point cloud coloring system is used for executing the method and the mobile platform comprises the point cloud coloring system.
In addition, the embodiment of the invention also provides a computer storage medium, and the computer storage medium is stored with the computer program. One or more computer program instructions may be stored on the computer-readable storage medium, which may be executed by a processor to execute the program instructions stored by the memory to implement the functions of the embodiments of the invention (implemented by the processor) described herein and/or other desired functions, such as to perform the corresponding steps of the point cloud coloring method according to the embodiments of the invention, and various applications and various data, such as various data used and/or generated by the applications, and the like, may also be stored in the computer-readable storage medium.
For example, the computer storage medium may include, for example, a memory card of a smart phone, a storage component of a tablet computer, a hard disk of a personal computer, a Read Only Memory (ROM), an Erasable Programmable Read Only Memory (EPROM), a portable compact disc read only memory (CD-ROM), a USB memory, or any combination of the above storage media. The computer-readable storage medium may be any combination of one or more computer-readable storage media. For example, a computer readable storage medium contains program code or the like for adjusting the reflectivity of the point cloud.
It should be understood that portions of the present application may be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. If implemented in hardware, as in another embodiment, any one or combination of the following techniques, which are known in the art, may be used: a discrete logic circuit having a logic Gate circuit for implementing a logic function on a data signal, an asic having a suitable combinational logic Gate circuit, a Programmable Gate Array (PGA), a Field Programmable Gate Array (FPGA), and the like.
Although the illustrative embodiments have been described herein with reference to the accompanying drawings, it is to be understood that the foregoing illustrative embodiments are merely exemplary and are not intended to limit the scope of the invention thereto. Various changes and modifications may be effected therein by one of ordinary skill in the pertinent art without departing from the scope or spirit of the present invention. All such changes and modifications are intended to be included within the scope of the present invention as set forth in the appended claims.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described device embodiments are merely illustrative, and for example, the division of the units is only one logical functional division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another device, or some features may be omitted, or not executed.
In the description provided herein, numerous specific details are set forth. It is understood, however, that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.
Similarly, it should be appreciated that in the description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the invention and aiding in the understanding of one or more of the various inventive aspects. However, the method of the present invention should not be construed to reflect the intent: that the invention as claimed requires more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single disclosed embodiment. Thus, the claims following the detailed description are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of this invention.
It will be understood by those skilled in the art that all of the features disclosed in this specification (including any accompanying claims, abstract and drawings), and all of the processes or elements of any method or apparatus so disclosed, may be combined in any combination, except combinations where such features are mutually exclusive. Each feature disclosed in this specification (including any accompanying claims, abstract and drawings) may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.
Furthermore, those skilled in the art will appreciate that while some embodiments described herein include some features included in other embodiments, rather than other features, combinations of features of different embodiments are meant to be within the scope of the invention and form different embodiments. For example, in the claims, any of the claimed embodiments may be used in any combination.
The various component embodiments of the invention may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art will appreciate that a microprocessor or Digital Signal Processor (DSP) may be used in practice to implement some or all of the functionality of some of the modules according to embodiments of the present invention. The present invention may also be embodied as apparatus programs (e.g., computer programs and computer program products) for performing a portion or all of the methods described herein. Such programs implementing the present invention may be stored on computer-readable media or may be in the form of one or more signals. Such a signal may be downloaded from an internet website or provided on a carrier signal or in any other form.
It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by one and the same item of hardware. The usage of the words first, second and third, etcetera do not indicate any ordering. These words may be interpreted as names.
Claims (37)
- A method of point cloud colorization, the method comprising:acquiring point cloud data, wherein the point cloud data comprises position information and reflectivity information of a point cloud;dividing the point cloud into at least two point clouds of position intervals according to the position information;respectively adjusting the reflectivity of the point clouds in each position interval to reduce the difference of the reflectivity of at least part of the point clouds in the same position interval;and coloring the point cloud according to the adjusted reflectivity.
- The method of claim 1, wherein the adjusting further comprises increasing a difference in reflectivity of the point cloud between different location intervals.
- The method of claim 1, wherein the segmenting the point cloud into at least two location intervals of point cloud according to the location information comprises:the method comprises the steps of determining a point cloud located in a target area, and dividing the point cloud in the target area into at least two point clouds of position intervals.
- The method of claim 3, wherein the target area comprises a road area.
- The method of claim 1 or 2, wherein the segmenting the point cloud into point clouds of at least two location intervals according to the location information comprises: separating ground points and ground points from the point cloud,the adjusting the reflectivity of the point cloud in each position interval comprises: expanding a difference between the reflectivity of the ground point and the reflectivity of the above-ground point.
- The method of claim 3, wherein the separately adjusting the reflectivity of the point cloud in each location interval further comprises:dividing the ground points into a plurality of first sub-location intervals;and respectively adjusting the reflectivity of the ground point in each first sub-position interval.
- The method of claim 6, wherein the first sub-location interval is segmented according to a distance between the point cloud and a ranging device that generated the point cloud.
- The method of claim 6 wherein the dividing the ground points into the ground points of the first plurality of sub-location intervals comprises: the ground points are divided into a plurality of two-dimensional mesh ground points.
- The method of claim 8, wherein each of the first sub-location intervals comprises at least one grid.
- The method according to any one of claims 6-9, wherein said individually adjusting the reflectivity of the ground points in each of said first sub-location intervals comprises:when the difference between the reflectivity of the ground points in the first sub-position interval does not exceed a preset threshold value, the reflectivity of the ground points in the first sub-position interval is adjusted according to the same trend; and/or the presence of a gas in the gas,adjusting the reflectivity of the ground points in the first sub-location interval according to different trends when the difference between the reflectivity of the ground points in the first sub-location interval exceeds a preset threshold.
- The method of claim 10 wherein said adjusting the reflectivity of the ground points in said first sub-interval of positions in different trends comprises:and adjusting the reflectivity of the ground points in the first sub-position interval by two value intervals respectively tending to the two ends of the reflectivity of the ground points in the first sub-position interval.
- The method of claim 11 wherein said individually adjusting the reflectivity of the ground points in each of said first sub-interval of positions further comprises:and carrying out convergence adjustment on the numerical value intervals with larger reflectivity of the ground points in different first sub-position intervals, and carrying out convergence adjustment on the numerical value intervals with smaller reflectivity of the ground points in different first sub-position intervals.
- The method of claim 5, wherein the separately adjusting the reflectivity of the point cloud in each location interval further comprises:dividing the above-ground point into at least two above-ground points of a second sub-position interval;and adjusting the reflectivity of the ground point in each second sub-position interval to reduce the difference of the reflectivities of the ground points in the same second sub-position interval and/or expand the difference of the reflectivities of the ground points between different second sub-position intervals.
- The method according to claim 13, wherein said dividing said above-ground point into above-ground points of at least two second sub-location intervals comprises:and dividing the ground point into at least two ground points of a second sub-position interval according to the height of the ground point above the plane of the ground point by taking the plane of the ground point as a reference.
- The method of claim 14 wherein the dividing the above-ground point into above-ground points of at least two second sub-location intervals according to the height of the above-ground point above the plane in which the ground point lies comprises:and determining the ground point in a first height above the plane where the ground point is located as the ground point of the first and second sub-position intervals, and determining the ground point between the first height and the second height above the plane where the ground point is located as the ground point of the second and second sub-position intervals.
- The method of claim 1, further comprising:and determining an adjusting coefficient according to the position information, and adjusting the reflectivity of the point cloud by using the adjusting coefficient, wherein the adjusting coefficients of the point cloud at least two different positions are different.
- The method of claim 16, wherein the location information comprises at least one of: distance information, height information, horizontal position information in the current field of view.
- The method of claim 1, wherein the acquiring point cloud data comprises:acquiring original point cloud data acquired by a distance measuring device, wherein the original point cloud data comprises pulse width information and distance information;calculating to obtain the reflectivity information according to the pulse width information and the distance information;and correcting the reflectivity information according to a pre-calibrated correction factor of the ranging device.
- A point cloud colorization system, the system comprising:a memory for storing executable instructions;a processor for executing the instructions stored in the memory, causing the processor to perform the steps of:acquiring point cloud data, wherein the point cloud data comprises position information and reflectivity information of a point cloud;dividing the point cloud into at least two point clouds of position intervals according to the position information;respectively adjusting the reflectivity of the point clouds in each position interval to reduce the difference of the reflectivity of at least part of the point clouds in the same position interval;and coloring the point cloud according to the adjusted reflectivity.
- The system of claim 19, wherein the adjusting further comprises increasing a difference in reflectivity of the point cloud between different location intervals.
- The system of claim 19, wherein the segmenting the point cloud into at least two location intervals of point cloud based on the location information comprises:the method comprises the steps of determining a point cloud located in a target area, and dividing the target area into at least two point clouds of position intervals.
- The system of claim 21, wherein the target area comprises a road area.
- The system of claim 19 or 20, wherein the segmenting the point cloud into at least two point clouds of location intervals according to the location information comprises: separating ground points and ground points from the point cloud,the adjusting the reflectivity of the point cloud comprises: expanding a difference between the reflectivity of the ground point and the reflectivity of the above-ground point.
- The system of claim 21, wherein the separately adjusting the reflectivity of the point cloud in each location interval further comprises:dividing the ground points into a plurality of first sub-location intervals;and respectively adjusting the reflectivity of the ground point in each first sub-position interval.
- The system of claim 24, wherein the first sub-location interval is segmented according to a distance between the point cloud and a ranging device that generated the point cloud.
- The system of claim 24 wherein the dividing the ground points into the ground points of the first plurality of sub-location intervals comprises: the ground points are divided into a plurality of two-dimensional mesh ground points.
- The system of claim 26, wherein each of the first sub-location intervals comprises at least one grid.
- The system according to any one of claims 24-27, wherein said individually adjusting the reflectivity of the ground points in each of said first sub-interval of positions comprises:when the difference between the reflectivity of the ground points in the first sub-position interval does not exceed a preset threshold value, the reflectivity of the ground points in the first sub-position interval is adjusted according to the same trend; and/or the presence of a gas in the gas,adjusting the reflectivity of the ground points in the first sub-location interval according to different trends when the difference between the reflectivity of the ground points in the first sub-location interval exceeds a preset threshold.
- The system of claim 28 wherein said adjusting the reflectivity of the ground points in said first sub-interval of locations according to different trends comprises:and adjusting the reflectivity of the ground points in the first sub-position interval by two value intervals respectively tending to the two ends of the reflectivity of the ground points in the first sub-position interval.
- The system of claim 29 wherein said individually adjusting the reflectivity of the ground points in each of said first sub-interval of positions further comprises:and carrying out convergence adjustment on the numerical value intervals with larger reflectivity of the ground points in different first sub-position intervals, and carrying out convergence adjustment on the numerical value intervals with smaller reflectivity of the ground points in different first sub-position intervals.
- The system of claim 23, wherein the separately adjusting the reflectivity of the point cloud in each location interval further comprises:dividing the above-ground point into at least two above-ground points of a second sub-position interval;and adjusting the reflectivity of the ground point in each second sub-position interval to reduce the difference of the reflectivities of the ground points in the same second sub-position interval and/or expand the difference of the reflectivities of the ground points between different second sub-position intervals.
- The system according to claim 31, wherein said dividing said above-ground point into at least two above-ground points of a second sub-location interval comprises:and dividing the ground point into at least two ground points of a second sub-position interval according to the height of the ground point above the plane of the ground point by taking the plane of the ground point as a reference.
- The system of claim 32 wherein said dividing said above-ground point into above-ground points of at least two second sub-location intervals according to the height of said above-ground point above the plane of said ground point comprises:and determining the ground point in a first height above the plane where the ground point is located as the ground point of the first and second sub-position intervals, and determining the ground point between the first height and the second height above the plane where the ground point is located as the ground point of the second and second sub-position intervals.
- The system of claim 19, wherein the processor is further configured to:and determining an adjusting coefficient according to the position information, and adjusting the reflectivity of the point cloud by using the adjusting coefficient, wherein the adjusting coefficients of the point cloud at least two different positions are different.
- The system of claim 34, wherein the location information comprises at least one of: distance information, height information, horizontal position information in the current field of view.
- The system of claim 19, wherein the acquiring point cloud data comprises:acquiring original point cloud data acquired by a distance measuring device, wherein the original point cloud data comprises pulse width information and distance information;calculating to obtain the reflectivity information according to the pulse width information and the distance information;and correcting the reflectivity information according to a pre-calibrated correction factor of the ranging device.
- A computer storage medium on which a computer program is stored, which program, when executed by a processor, implements the point cloud colorization method of any one of claims 1 to 18.
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| PCT/CN2020/091091 WO2021232247A1 (en) | 2020-05-19 | 2020-05-19 | Point cloud coloring method, point cloud coloring system, and computer storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114026410A true CN114026410A (en) | 2022-02-08 |
Family
ID=78709019
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202080005200.XA Pending CN114026410A (en) | 2020-05-19 | 2020-05-19 | Point cloud coloring method, point cloud coloring system, and computer storage medium |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN114026410A (en) |
| WO (1) | WO2021232247A1 (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118091685A (en) * | 2024-04-23 | 2024-05-28 | 武汉大学 | High-frequency high-precision color point cloud sensor and working method |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114529652B (en) * | 2022-04-24 | 2022-07-19 | 深圳思谋信息科技有限公司 | Point cloud compensation method, device, equipment and storage medium |
| CN115049826A (en) * | 2022-05-18 | 2022-09-13 | 洛伦兹(宁波)科技有限公司 | Lane line detection method, device and system |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6686921B1 (en) * | 2000-08-01 | 2004-02-03 | International Business Machines Corporation | Method and apparatus for acquiring a set of consistent image maps to represent the color of the surface of an object |
| CN104374780A (en) * | 2014-12-04 | 2015-02-25 | 上海岩土工程勘察设计研究院有限公司 | In-tunnel three-dimensional laser scanning reflectivity correction method |
| CN107851176A (en) * | 2015-02-06 | 2018-03-27 | 阿克伦大学 | Optical imaging system and its method |
| CN109191553A (en) * | 2018-08-29 | 2019-01-11 | 百度在线网络技术（北京）有限公司 | point cloud rendering method, device, terminal and storage medium |
| CN109506672A (en) * | 2017-09-15 | 2019-03-22 | 高德软件有限公司 | A kind of acquisition methods and device of pavement markers laser point cloud |
| CN110031822A (en) * | 2019-04-22 | 2019-07-19 | 上海禾赛光电科技有限公司 | It can be used for noise recognition methods and the laser radar system of laser radar |
| CN110132130A (en) * | 2019-03-05 | 2019-08-16 | 上海宾通智能科技有限公司 | Laser radar position calibration method, system and its data processing method, system |
| US20190391270A1 (en) * | 2018-06-25 | 2019-12-26 | Toyota Motor Engineering & Manufacturing North America, Inc. | Systems and methods for mitigating effects of high-reflectivity objects in lidar data |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP3259615B1 (en) * | 2015-02-20 | 2019-11-20 | Apple Inc. | Actuated optical element for light beam scanning device |
-
2020
- 2020-05-19 WO PCT/CN2020/091091 patent/WO2021232247A1/en active Application Filing
- 2020-05-19 CN CN202080005200.XA patent/CN114026410A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6686921B1 (en) * | 2000-08-01 | 2004-02-03 | International Business Machines Corporation | Method and apparatus for acquiring a set of consistent image maps to represent the color of the surface of an object |
| CN104374780A (en) * | 2014-12-04 | 2015-02-25 | 上海岩土工程勘察设计研究院有限公司 | In-tunnel three-dimensional laser scanning reflectivity correction method |
| CN107851176A (en) * | 2015-02-06 | 2018-03-27 | 阿克伦大学 | Optical imaging system and its method |
| CN109506672A (en) * | 2017-09-15 | 2019-03-22 | 高德软件有限公司 | A kind of acquisition methods and device of pavement markers laser point cloud |
| US20190391270A1 (en) * | 2018-06-25 | 2019-12-26 | Toyota Motor Engineering & Manufacturing North America, Inc. | Systems and methods for mitigating effects of high-reflectivity objects in lidar data |
| CN109191553A (en) * | 2018-08-29 | 2019-01-11 | 百度在线网络技术（北京）有限公司 | point cloud rendering method, device, terminal and storage medium |
| CN110132130A (en) * | 2019-03-05 | 2019-08-16 | 上海宾通智能科技有限公司 | Laser radar position calibration method, system and its data processing method, system |
| CN110031822A (en) * | 2019-04-22 | 2019-07-19 | 上海禾赛光电科技有限公司 | It can be used for noise recognition methods and the laser radar system of laser radar |
Non-Patent Citations (1)
| Title |
|---|
| 叶珉吕;花向红;: "顾及激光反射率的点云数据分割算法研究", 城市勘测, no. 06, 31 December 2014 (2014-12-31) * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118091685A (en) * | 2024-04-23 | 2024-05-28 | 武汉大学 | High-frequency high-precision color point cloud sensor and working method |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2021232247A1 (en) | 2021-11-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP7297017B2 (en) | Method and apparatus for calibrating external parameters of on-board sensors and related vehicles | |
| CN114026410A (en) | Point cloud coloring method, point cloud coloring system, and computer storage medium | |
| WO2022126427A1 (en) | Point cloud processing method, point cloud processing apparatus, mobile platform, and computer storage medium | |
| EP4130798A1 (en) | Target identification method and device | |
| WO2020243962A1 (en) | Object detection method, electronic device and mobile platform | |
| CN114080625A (en) | Absolute pose determination method, electronic equipment and movable platform | |
| CN112912756A (en) | Point cloud noise filtering method, distance measuring device, system, storage medium and mobile platform | |
| CN108414998B (en) | Satellite laser altimeter echo waveform analog simulation method and device | |
| CA3088983A1 (en) | Apparatuses and methods for gas flux measurements | |
| CN111902730B (en) | Calibration plate, depth parameter calibration method, detection device and calibration system | |
| CN112204568A (en) | Pavement mark recognition method and device | |
| US20240103142A1 (en) | Space measurement apparatus, method, and device, and computer-readable storage medium | |
| WO2022198637A1 (en) | Point cloud noise filtering method and system, and movable platform | |
| WO2017098934A1 (en) | Laser measuring system and laser measuring method | |
| CN114442073A (en) | Laser radar calibration method and device, vehicle and storage medium | |
| CN112136018A (en) | Point cloud noise filtering method of distance measuring device, distance measuring device and mobile platform | |
| JP2023111952A (en) | Sensor control device | |
| US20210255289A1 (en) | Light detection method, light detection device, and mobile platform | |
| CN114026461A (en) | Method for constructing point cloud frame, target detection method, distance measuring device, movable platform and storage medium | |
| WO2020237663A1 (en) | Multi-channel lidar point cloud interpolation method and ranging apparatus | |
| WO2020177076A1 (en) | Detection apparatus initial-state calibration method and apparatus | |
| WO2022256976A1 (en) | Method and system for constructing dense point cloud truth value data and electronic device | |
| CN115824170A (en) | Method for measuring ocean waves by combining photogrammetry and laser radar | |
| KR101232292B1 (en) | Calculating system for open area ratio of the sky using aerial lidar data | |
| CN111801707A (en) | Method, device and system for point cloud resampling |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |