CN114265035A

CN114265035A - Laser radar data processing method and system

Info

Publication number: CN114265035A
Application number: CN202111475271.9A
Authority: CN
Inventors: 刘旭; 孙冬
Original assignee: Beijing Jingwei Hirain Tech Co Ltd
Current assignee: Beijing Jingwei Hirain Tech Co Ltd
Priority date: 2021-12-06
Filing date: 2021-12-06
Publication date: 2022-04-01

Abstract

The application discloses a laser radar data processing method and a system, wherein the method comprises the following steps: acquiring a multi-frame two-dimensional depth map converted from laser radar original data and a filter determined according to the ratio of the transverse angle resolution to the longitudinal angle resolution of the laser radar; carrying out intraframe compression on each frame of two-dimensional depth map according to a filter and a preset plane fitting algorithm to obtain a multi-frame intraframe compressed two-dimensional depth map; for a two-dimensional depth map group, performing interframe compression on the two-dimensional depth map group according to the difference between the two-dimensional depth map compressed in the second frame to the two-dimensional depth map compressed in the last frame in the two-dimensional depth map group and the two-dimensional depth map compressed in the first frame; and after all the two-dimensional depth map groups are subjected to interframe compression, all the two-dimensional depth maps subjected to interframe compression are subjected to overall compression to obtain the two-dimensional depth maps subjected to overall compression. According to the data compression method and device, data compression can be achieved from multiple angles, and therefore the data volume is reduced.

Description

Laser radar data processing method and system

Technical Field

The application relates to the technical field of internet, in particular to a laser radar data processing method and system.

Background

The laser radar is a radar system that detects a characteristic amount such as a position and a velocity of a target by emitting a laser beam. The working principle of the system is that a detection signal (namely a laser beam) is firstly transmitted to a target, then a received signal (namely a target echo) reflected from the target is compared with the transmitted signal, and after appropriate processing, relevant information of the target, such as target distance, direction, height, speed, attitude, even shape and other parameters, can be obtained, so that the detection, tracking and identification of the targets such as objects around a vehicle, airplanes, missiles and the like are realized.

The laser radar is one of the mainstream vehicle-mounted sensors at present, has the advantages of high measurement precision, long measurement distance, high response speed, easiness in obtaining three-dimensional information and the like, and is often used as a main sensor of vehicle-end and road-side sensing or other intelligent equipment. Meanwhile, due to the characteristics of high precision and large amount of information of the laser point cloud, in application scenes such as laser radar data transmission and storage, huge pressure is brought to a transmission and storage link of a system by massive data. Therefore, how to reduce the data volume of the storage or transmission of the lidar data is an important problem to be solved urgently.

Disclosure of Invention

The application provides a laser radar data processing method and system, which can perform intraframe compression, interframe compression and overall compression on a two-dimensional depth map converted from laser radar original data, thereby greatly reducing the data volume.

The specific technical scheme is as follows:

in a first aspect, an embodiment of the present application provides a method for processing laser radar data, where the method includes:

acquiring a multi-frame two-dimensional depth map converted from original data of a laser radar, and acquiring a filter determined according to the ratio of the transverse angle resolution to the longitudinal angle resolution of the laser radar;

carrying out intraframe compression on each frame of two-dimensional depth map according to the filter and a preset plane fitting algorithm to obtain a plurality of frames of intraframe compressed two-dimensional depth maps;

for a two-dimensional depth map group, performing interframe compression on the two-dimensional depth map group according to the difference between the two-dimensional depth map compressed in the second frame to the last frame in the two-dimensional depth map group and the two-dimensional depth map compressed in the first frame, wherein each two-dimensional depth group comprises the two-dimensional depth map compressed in the continuous multi-frame with the frame number of a preset frame number;

and after all the two-dimensional depth map groups are subjected to interframe compression, all the two-dimensional depth maps subjected to interframe compression are subjected to overall compression to obtain the two-dimensional depth maps subjected to overall compression.

In one embodiment, obtaining a filter determined from a ratio of a transverse angular resolution and a longitudinal angular resolution of a lidar includes: obtaining a filter with the height-width ratio being the ratio of the transverse angular resolution to the longitudinal angular resolution of the laser radar;

before each frame of two-dimensional depth map is subjected to intraframe compression according to the filter and a preset plane fitting algorithm to obtain a multi-frame intraframe compressed two-dimensional depth map, the method further comprises the following steps: if the compression ratio and/or the signal-to-noise ratio of the filter with the height-width ratio being the ratio of the transverse angular resolution to the longitudinal angular resolution of the laser radar do not meet the preset compression requirement, the size of the filter is adjusted according to the transverse distance resolution of the laser radar at the interested distance, the longitudinal distance resolution of the laser radar at the interested distance and the size of the target object at the interested distance, and the finally required filter is obtained until the compression ratio and the signal-to-noise ratio of the adjusted filter meet the preset compression requirement, wherein the interested distance is the distance corresponding to the original data of the laser radar, and the size comprises the height and the width.

In one embodiment, resizing the filter according to a lateral range resolution of the lidar at a range of interest, a longitudinal range resolution at the range of interest, and a size of a target object at the range of interest includes:

calculating a ratio of a height of a target object at the distance of interest to the lateral range resolution of the lidar at the distance of interest, a ratio of a width of the target object to the longitudinal range resolution, respectively;

if the ratio of the height of the target object to the transverse distance resolution is larger than the ratio of the width of the target object to the longitudinal distance resolution, the size occupied by the height of the filter is increased;

if the ratio of the height of the target object to the transverse distance resolution is smaller than the ratio of the width of the target object to the longitudinal distance resolution, the size occupied by the width of the filter is increased.

In one embodiment, the intra-frame compression is performed on each frame of two-dimensional depth map according to the filter and a preset plane fitting algorithm, so as to obtain a multi-frame intra-frame compressed two-dimensional depth map, and the method includes:

performing plane fitting on a first target area on the to-be-intra compressed two-dimensional depth map filtered by the filter according to the preset plane fitting algorithm to obtain a first fitting plane;

calculating a first fitting error according to the difference between the coordinates of the plurality of data points on the first fitting plane and the coordinates of the same corresponding data point on the two-dimensional depth map to be intra-frame compressed;

if the first fitting error is smaller than a preset error threshold, keeping a first fitting record, wherein the first fitting record comprises a fitting position of the first fitting plane and a fitting parameter of the first fitting plane, the fitting position of the first fitting plane is a position of the first target area on the compressed two-dimensional depth map in the frame to be compressed, and the fitting parameter of the first fitting plane comprises a normal vector and a geometric equation of the first fitting plane;

if the first fitting error is larger than or equal to the preset error threshold, not keeping the first fitting record;

after the filter is slid to a second target area, performing plane fitting on the first target area and the second target area as an integral area according to the preset plane fitting algorithm to obtain a second fitting plane;

calculating a second fitting error according to the difference between the coordinates of the plurality of data points on the second fitting plane and the coordinates of the same corresponding data point on the to-be-intraframe compressed two-dimensional depth map before plane fitting;

if the second fitting error is smaller than the preset error threshold, replacing the first fitting record with a second fitting record, wherein the second fitting record comprises a fitting position of the second fitting plane and fitting parameters of the second fitting plane, the fitting position of the second fitting plane comprises a position of the first target area and a position of the second target area, and the fitting parameters of the second fitting plane comprise a normal vector and a geometric equation of the second fitting plane;

if the second fitting error is larger than or equal to the preset error threshold, performing plane fitting on the second target area again until the last target area of the to-be-intra-frame compressed two-dimensional depth map is subjected to plane fitting processing, and obtaining the to-be-intra-frame compressed two-dimensional depth map, wherein the to-be-intra-frame compressed two-dimensional depth map comprises the last reserved fitting record and original data of the target area, on which the fitting record is not reserved, of the to-be-intra-frame compressed two-dimensional depth map;

and after the intraframe compression is completed on the multiple frames of two-dimensional depth maps, obtaining the multiple frames of intraframe compressed two-dimensional depth maps.

In one embodiment, for a two-dimensional depth map packet, inter-frame compression is performed on the two-dimensional depth map packet according to differences between two-dimensional depth maps compressed in a second frame and a last frame of the two-dimensional depth map packet and two-dimensional depth maps compressed in a first frame of the two-dimensional depth map packet, including:

for each frame of intra-compressed two-dimensional depth map except for a first frame of intra-compressed two-dimensional depth map in a two-dimensional depth map group, respectively calculating differences between a plurality of target areas in the current frame of intra-compressed two-dimensional depth map and target areas corresponding to the first frame of intra-compressed two-dimensional depth map, and obtaining inter-frame deviation of the current frame of intra-compressed two-dimensional depth map relative to the first frame of intra-compressed two-dimensional depth map, wherein the target areas are the size of areas subjected to primary filtering by the filter;

and reserving the compressed two-dimensional depth map in the first frame, and replacing the compressed two-dimensional depth map in the current frame with the position of the compressed two-dimensional depth map in the current frame and the corresponding inter-frame deviation.

In one embodiment, acquiring a multi-frame two-dimensional depth map into which raw lidar data is converted includes:

receiving laser radar original data sent by the laser radar;

converting the laser radar original data into multi-frame laser radar point cloud data;

sequentially adding each frame of laser radar point cloud data into a point cloud first-in first-out (FIFO) queue according to the conversion sequence of the laser radar original data;

sequentially acquiring each frame of laser radar point cloud data from the point cloud FIFO queue according to the principle of first-in first-out of the point cloud FIFO queue, and converting each frame of laser radar point cloud data into a two-dimensional depth map;

sequentially adding the converted two-dimensional depth maps into a depth map FIFO queue according to the conversion sequence of the laser radar point cloud data;

and sequentially acquiring each frame of laser radar two-dimensional depth map in the depth map FIFO queue according to the principle of first-in first-out of the depth map FIFO queue.

In one embodiment, after performing inter-frame compression on all the two-dimensional depth map packets, performing overall compression on all the inter-compressed two-dimensional depth maps to obtain an overall compressed two-dimensional depth map, including:

sequentially adding the two-dimensional depth groups after the interframe compression to a data transmission FIFO queue according to the interframe compression sequence of the two-dimensional depth map groups;

and after adding the two-dimensional depth groups after all interframes are compressed into the data transmission FIFO queue, integrally compressing the data transmission FIFO queue to obtain an integrally compressed two-dimensional depth map.

In one embodiment, after performing overall compression on all the inter-compressed two-dimensional depth maps to obtain an overall compressed two-dimensional depth map, the method includes:

when the integrally compressed two-dimensional depth map needs to be decompressed, determining the integrally compressed two-dimensional depth map as a two-dimensional depth map to be decompressed;

integrally decompressing the two-dimensional depth map to be decompressed to obtain a plurality of two-dimensional depth map groups to be decompressed among frames, wherein the two-dimensional depth map groups to be decompressed among frames are two-dimensional depth map groups obtained by performing inter-frame compression on continuous multi-frame intra-frame compressed two-dimensional depth maps with preset frame numbers;

according to the difference between the two-dimensional depth map after the compression in the second frame to the two-dimensional depth map after the compression in the last frame in the two-dimensional depth map group to be subjected to inter-frame decompression and the two-dimensional depth map after the compression in the first frame, performing inter-frame decompression on the two-dimensional depth map group to be subjected to inter-frame decompression to obtain a plurality of frames of two-dimensional depth maps to be subjected to intra-frame decompression, wherein the two-dimensional depth maps to be subjected to intra-frame decompression comprise at least one fitting position, fitting parameters corresponding to the fitting position and original data of the two-dimensional depth map to be subjected to intra-frame decompression at the position which is not fitted before the intra-frame compression, and the fitting parameters comprise a normal vector of a fitting plane and a geometric equation of the fitting plane;

and for each frame of two-dimensional depth map to be subjected to intra-frame decompression, performing intra-frame decompression on the fitting position of the two-dimensional depth map to be subjected to intra-frame decompression according to the fitting position and the fitting parameters corresponding to the fitting position, and obtaining original data of the fitting position on the two-dimensional depth map subjected to intra-frame decompression, so that when all original data of the two-dimensional depth map to be subjected to intra-frame decompression on the two-dimensional depth map subjected to intra-frame decompression are obtained, the two-dimensional depth map subjected to intra-frame decompression is obtained.

In one embodiment, when the to-be-inter-frame decompressed two-dimensional depth map packet includes an inter-frame deviation between a two-dimensional depth map compressed in a first frame of the to-be-inter-frame decompressed two-dimensional depth map packet, a position of a two-dimensional depth map compressed in other frames except the first frame of the to-be-inter-frame decompressed two-dimensional depth map packet, and a position of the two-dimensional depth map compressed in other frames relative to the two-dimensional depth map compressed in the first frame of the to-be-inter-frame decompressed two-dimensional depth map packet, inter-frame decompression is performed on the to-be-inter-frame decompressed two-dimensional depth map packet according to a difference between the two-dimensional depth map compressed in a second frame of the to a last frame of the to-be-inter-frame decompressed two-dimensional depth map packet and the two-dimensional depth map compressed in the first frame of the to-be-frame decompressed two-dimensional depth map packet, so as to obtain a multi-frame to-be-frame intra-frame two-dimensional depth map, the inter-frame decompressed two-frame depth map packet includes:

for each frame of intra-frame compressed two-dimensional depth map except a first frame of intra-frame compressed two-dimensional depth map in the two-dimensional depth map group to be inter-frame decompressed, determining the current frame of intra-frame compressed two-dimensional depth map according to the position of the current frame of intra-frame compressed two-dimensional depth map, the inter-frame deviation of the current frame of intra-frame compressed two-dimensional depth map relative to the first frame of intra-frame compressed two-dimensional depth map, and the first frame of intra-frame compressed two-dimensional depth map;

after the two-dimensional depth map compressed in each frame of the two-dimensional depth map group to be decompressed among frames is determined, the two-dimensional depth map compressed in the frames is determined to be the two-dimensional depth map to be decompressed among the frames, so that the two-dimensional depth map to be decompressed among the frames is obtained.

In a second aspect, an embodiment of the present application provides an apparatus for processing lidar data, where the apparatus includes:

the device comprises a depth map acquisition unit, a processing unit and a processing unit, wherein the depth map acquisition unit is used for acquiring a multi-frame two-dimensional depth map converted from original data of the laser radar;

the laser radar device comprises a filter acquisition unit, a filter processing unit and a control unit, wherein the filter acquisition unit is used for acquiring a filter determined according to the ratio of the transverse angle resolution to the longitudinal angle resolution of the laser radar;

the intraframe compression unit is used for performing intraframe compression on each frame of two-dimensional depth map according to the filter and a preset plane fitting algorithm to obtain a plurality of frames of intraframe compressed two-dimensional depth maps;

the inter-frame compression unit is used for performing inter-frame compression on two-dimensional depth map groups according to the difference between the two-dimensional depth maps which are compressed in the second frame and the last frame in the two-dimensional depth map groups and the two-dimensional depth maps which are compressed in the first frame, wherein each two-dimensional depth group comprises the two-dimensional depth maps which are compressed in the continuous multi-frame with the number of frames being a preset number of frames;

and the integral compression unit is used for integrally compressing all the two-dimensional depth maps after the inter-frame compression is carried out on all the two-dimensional depth map groups to obtain the integrally compressed two-dimensional depth maps.

In one embodiment, the depth map acquisition unit is used for acquiring a filter with the ratio of height to width being the ratio of the transverse angular resolution to the longitudinal angular resolution of the laser radar;

the device further comprises:

an adjusting unit, configured to, before performing intra-frame compression on each frame of two-dimensional depth map according to the filter and a preset plane fitting algorithm to obtain a multi-frame intra-compressed two-dimensional depth map, if a compression ratio and/or a signal-to-noise ratio of the filter, where a ratio of a height to a width is a ratio of a lateral angular resolution to a longitudinal angular resolution of the laser radar, does not meet a preset compression requirement, adjust a size of the filter according to a lateral distance resolution of the laser radar at a distance of interest, a longitudinal distance resolution at the distance of interest, and a size of a target object at the distance of interest, until the compression ratio and the signal-to-noise ratio of the adjusted filter meet the preset compression requirement, obtain a finally required filter, where the distance of interest is a distance corresponding to original data of the laser radar, the dimensions include height and width.

In one embodiment, the adjusting unit includes:

a ratio calculation module for calculating a ratio of a height of a target object at the distance of interest to the transverse range resolution of the lidar at the distance of interest, and a ratio of a width of the target object to the longitudinal range resolution, respectively;

an adjusting module, configured to increase a size proportion occupied by the height of the filter if a ratio of the height of the target object to the lateral distance resolution is greater than a ratio of the width of the target object to the longitudinal distance resolution, and increase a size proportion occupied by the width of the filter if the ratio of the height of the target object to the lateral distance resolution is less than the ratio of the width of the target object to the longitudinal distance resolution.

In one embodiment, an intra-frame compression unit includes:

the fitting module is used for performing plane fitting on a first target area on the to-be-intra-frame compression two-dimensional depth map filtered by the filter according to the preset plane fitting algorithm to obtain a first fitting plane;

the error calculation module is used for calculating a first fitting error according to the difference between the coordinates of a plurality of data points on the first fitting plane and the coordinates of the same corresponding data point on the two-dimensional depth map to be intra-frame compressed;

a first retaining module, configured to retain a first fitting record if the first fitting error is smaller than a preset error threshold, where the first fitting record includes a fitting position of the first fitting plane and a fitting parameter of the first fitting plane, the fitting position of the first fitting plane is a position of the first target region on the compressed two-dimensional depth map in the frame to be compressed, the fitting parameter of the first fitting plane includes a normal vector and a geometric equation of the first fitting plane, and if the first fitting error is greater than or equal to the preset error threshold, the first fitting record is not retained;

the fitting module is used for performing plane fitting on the first target area and the second target area as an integral area according to the preset plane fitting algorithm after the filter is slid to the second target area, so as to obtain a second fitting plane;

the error calculation module is used for calculating a second fitting error according to the difference between the coordinates of a plurality of data points on the second fitting plane and the coordinates of the same corresponding data point on the to-be-intra compressed two-dimensional depth map before plane fitting;

a first replacement module, configured to replace the first fitting record with a second fitting record if the second fitting error is smaller than the preset error threshold, where the second fitting record includes a fitting position of the second fitting plane and a fitting parameter of the second fitting plane, the fitting position of the second fitting plane includes a position of the first target area and a position of the second target area, and the fitting parameter of the second fitting plane includes a normal vector and a geometric equation of the second fitting plane;

the fitting module is configured to perform plane fitting on the second target area again if the second fitting error is greater than or equal to the preset error threshold value, until a last target area of the to-be-intra-frame compressed two-dimensional depth map is subjected to plane fitting processing, and obtain a to-be-intra-frame compressed two-dimensional depth map, wherein the to-be-intra-frame compressed two-dimensional depth map includes a last reserved fitting record and original data of a target area of which the fitting record is not reserved on the to-be-intra-frame compressed two-dimensional depth map;

and the first acquisition module is used for acquiring the two-dimensional depth map after the multi-frame two-dimensional depth map is subjected to intra-frame compression.

In one embodiment, an inter-frame compression unit includes:

a deviation calculation module, configured to calculate, for each frame of intra-frame compressed two-dimensional depth map except for a first frame of intra-frame compressed two-dimensional depth map in a two-dimensional depth map group, differences between a plurality of target areas in the current frame of intra-frame compressed two-dimensional depth map and a target area corresponding to the first frame of intra-frame compressed two-dimensional depth map, to obtain inter-frame deviation of the current frame of intra-frame compressed two-dimensional depth map with respect to the first frame of intra-frame compressed two-dimensional depth map, where the target area is an area size of a filter performing primary filtering;

a second reserving module, configured to reserve the compressed two-dimensional depth map in the first frame;

and the second replacing module is used for replacing the compressed two-dimensional depth map in the current frame with the position of the compressed two-dimensional depth map in the current frame and the corresponding inter-frame deviation.

In one embodiment, the depth map acquisition unit includes:

the receiving module is used for receiving laser radar original data sent by the laser radar;

the first conversion module is used for converting the laser radar original data into multi-frame laser radar point cloud data;

the first adding module is used for sequentially adding each frame of laser radar point cloud data into a point cloud first-in first-out FIFO queue according to the conversion sequence of the laser radar original data;

the second conversion module is used for sequentially acquiring each frame of laser radar point cloud data from the point cloud FIFO queue according to the principle of first-in first-out of the point cloud FIFO queue and converting each frame of laser radar point cloud data into a two-dimensional depth map;

the second adding module is used for sequentially adding the converted two-dimensional depth maps into the depth map FIFO queue according to the conversion sequence of the point cloud data of the laser radar;

and the second acquisition module is used for sequentially acquiring each frame of laser radar two-dimensional depth map in the depth map FIFO queue according to the principle of first-in first-out of the depth map FIFO queue.

In one embodiment, an integral compression unit, comprising:

the third adding module is used for sequentially adding the two-dimensional depth groups after the interframe compression to the data transmission FIFO queue according to the interframe compression sequence of the two-dimensional depth map groups;

and the integral compression module is used for integrally compressing the data transmission FIFO queue after the two-dimensional depth groups compressed between all frames are added into the data transmission FIFO queue to obtain an integrally compressed two-dimensional depth map.

In one embodiment, the apparatus further comprises:

the device comprises a determining unit, a calculating unit and a calculating unit, wherein the determining unit is used for determining the two-dimensional depth map after the integral compression as a two-dimensional depth map to be decompressed when the two-dimensional depth map after the integral compression needs to be decompressed after all the two-dimensional depth maps after the inter-compression are compressed integrally to obtain the two-dimensional depth map after the integral compression;

the integral decompression unit is used for integrally decompressing the two-dimensional depth map to be decompressed to obtain a plurality of two-dimensional depth map groups to be decompressed among frames, wherein the two-dimensional depth map groups to be decompressed among frames are two-dimensional depth map groups obtained by compressing the two-dimensional depth map after the two-dimensional depth map is compressed among continuous multi-frame frames with the frame number being a preset frame number;

the inter-frame decompression unit is used for performing inter-frame decompression on the to-be-inter-frame decompressed two-dimensional depth map group according to the difference between the two-dimensional depth map compressed in the second frame to the last frame in the to-be-inter-frame decompressed two-dimensional depth map group and the two-dimensional depth map compressed in the first frame to obtain a multi-frame to-be-intra-frame decompressed two-dimensional depth map, wherein the to-be-intra-frame decompressed two-dimensional depth map comprises at least one fitting position, fitting parameters corresponding to the fitting position and original data of the to-be-intra-frame decompressed two-dimensional depth map at a position which is not fitted before intra-frame compression, and the fitting parameters comprise a normal vector of a fitting plane and a geometric equation of the fitting plane;

and the intra-frame decompression unit is used for performing intra-frame decompression on the fitting position of the two-dimensional depth map to be subjected to intra-frame decompression according to the fitting position and the fitting parameters corresponding to the fitting position, so as to obtain the original data of the fitting position on the two-dimensional depth map subjected to intra-frame decompression, and obtain the two-dimensional depth map subjected to intra-frame decompression when all the original data of the two-dimensional depth map to be subjected to intra-frame decompression on the two-dimensional depth map subjected to intra-frame decompression are obtained.

In one embodiment, an inter-frame decompression unit includes:

an inter-frame decompression module, configured to, when the to-be-inter-frame decompressed two-dimensional depth map packet includes the first intra-frame compressed two-dimensional depth map, the positions of the other intra-frame compressed two-dimensional depth maps except the first intra-frame compressed two-dimensional depth map, and the inter-frame deviations of the positions of the other intra-frame compressed two-dimensional depth maps with respect to the first intra-frame compressed two-dimensional depth map, for the each intra-frame compressed two-dimensional depth map except the first intra-frame compressed two-dimensional depth map in the to-be-inter-frame decompressed two-dimensional depth map packet, for the position of the compressed two-dimensional depth map in the current frame, the inter-frame deviations of the compressed two-dimensional depth map in the current frame with respect to the first intra-frame compressed two-dimensional depth map, and the first intra-frame compressed two-dimensional depth map, determining a compressed two-dimensional depth map in the current frame;

and the determining module is used for determining the two-dimensional depth map compressed in each frame of frame in the two-dimensional depth map group to be decompressed among frames as the two-dimensional depth map to be decompressed in frame so as to obtain the two-dimensional depth map to be decompressed in frame.

In a third aspect, an embodiment of the present application provides a system for processing lidar data, where the system includes a lidar, a terminal device, and a server;

the laser radar is used for acquiring laser radar original data and sending the laser radar original data to the terminal equipment;

the terminal device is configured to obtain an overall compressed two-dimensional depth map by executing the method according to any one of the embodiments of the first aspect, and send the overall compressed two-dimensional depth map to the server;

the server is used for receiving the integrally compressed two-dimensional depth map sent by the terminal device, determining the integrally compressed two-dimensional depth map as a two-dimensional depth map to be decompressed, integrally decompressing the two-dimensional depth map to be decompressed, obtaining a plurality of two-dimensional depth map groups to be decompressed among frames, wherein the two-dimensional depth map groups to be decompressed among frames are two-dimensional depth map groups compressed among frames which are preset and continuous and are compressed in multiple frames, and inter-frame decompressing the two-dimensional depth map groups to be decompressed among frames according to the difference between the two-dimensional depth maps compressed in the second frame to the last frame in the two-dimensional depth map groups to be decompressed among frames and the two-dimensional depth map compressed in the first frame, so as to obtain the two-dimensional depth maps in multiple frames to be decompressed, wherein the two-dimensional depth map to be intra-frame decompressed comprises at least one fitting position, fitting parameters corresponding to the fitting position and original data of the two-dimensional depth map to be intra-frame decompressed at a position which is not fitted before intra-frame compression, the fitting parameters comprise a normal vector of a fitting plane and a geometric equation of the fitting plane, aiming at the two-dimensional depth map to be decompressed in each frame, according to the fitting position and the fitting parameters corresponding to the fitting position, performing intra-frame decompression on the fitting position of the two-dimensional depth map to be subjected to intra-frame decompression to obtain original data of the fitting position on the two-dimensional depth map subjected to intra-frame decompression, so that when all the original data of the two-dimensional depth map to be intra-decompressed on the intra-decompressed two-dimensional depth map are obtained, and obtaining the two-dimensional depth map after the intra-frame decompression is carried out on the two-dimensional depth map to be subjected to intra-frame decompression.

In one embodiment, the server is configured to, when the two-dimensional depth map packet to be inter-frame decompressed includes a first intra-frame compressed two-dimensional depth map, a position of a second intra-frame compressed two-dimensional depth map other than the first intra-frame compressed two-dimensional depth map, and an inter-frame deviation of the position of the second intra-frame compressed two-dimensional depth map with respect to the first intra-frame compressed two-dimensional depth map in the two-dimensional depth map packet to be inter-frame decompressed, for each intra-frame compressed two-dimensional depth map other than the first intra-frame compressed two-dimensional depth map in the two-dimensional depth map packet to be inter-frame decompressed, for the position of the second intra-frame compressed two-dimensional depth map, the inter-frame deviation of the second intra-frame compressed two-dimensional depth map with respect to the first intra-frame compressed two-dimensional depth map, and the first intra-frame compressed two-frame depth map in the two-frame decompressed two-frame depth map, and determining the compressed two-dimensional depth map in the current frame, and determining the compressed two-dimensional depth map in each frame in the two-dimensional depth map group to be decompressed among frames as the two-dimensional depth map to be decompressed in the frame so as to obtain the multi-frame two-dimensional depth map to be decompressed in the frame.

In a fourth aspect, the present application provides a storage medium having stored thereon executable instructions, which when executed by a processor, cause the processor to implement the method according to any one of the embodiments of the first aspect.

In a fifth aspect, an embodiment of the present application provides an electronic device, including:

one or more processors;

a storage device for storing one or more programs,

wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the method of any of the embodiments of the first aspect.

As can be seen from the above, the method and system for processing lidar data provided in the embodiments of the present application can obtain a multi-frame two-dimensional depth map converted from lidar raw data, obtain a filter determined according to a ratio of a transverse angular resolution to a longitudinal angular resolution of the lidar, perform intra-frame compression on each frame of the two-dimensional depth map according to the filter and a preset plane fitting algorithm, obtain a multi-frame intra-frame compressed two-dimensional depth map, perform inter-frame compression on a two-dimensional depth map packet composed of consecutive multi-frame intra-frame compressed two-dimensional depth maps with a frame number of a preset frame number, and finally perform overall compression on all the two-dimensional depth maps after the inter-frame compression is completed, thereby obtaining a final compression result. Therefore, the method and the device can realize intraframe compression, interframe compression and overall compression of the two-dimensional depth map converted from the original data of the laser radar, greatly reduce the data volume of the two-dimensional depth map from the aspect of multi-angle compression, and filter the two-dimensional depth map according to the filter determined by the transverse and longitudinal angle resolution, but not directly use the filter with the same width and height to filter the two-dimensional depth map, thereby improving the compression rate and the compression speed. When laser radar data (namely laser radar original data or a two-dimensional depth map) needs to be stored locally, storage space can be saved according to the embodiment of the application, and transmission efficiency can be improved when the laser radar data need to be transmitted outwards.

The embodiment of the application can also achieve the technical effects of at least:

1. if the compression ratio and/or the signal-to-noise ratio of the filter, of which the height-to-width ratio is the ratio of the transverse angle resolution to the longitudinal angle resolution of the laser radar, do not meet the preset compression requirement, the size of the filter can be rapidly adjusted through the transverse distance resolution, the longitudinal distance resolution and the size of the target object at the interested distance of the laser radar, blind adjustment is not needed, and therefore the adjustment efficiency can be improved.

2. During data format conversion (including conversion of laser radar original data into laser radar point cloud data and conversion of the laser radar point cloud data into a two-dimensional depth map) and data compression, writing and reading can be performed by adopting different FIFO (First Input First Output) queues, and through simultaneous operation of multiple threads, the frame rate of laser radar data processed by an algorithm can be improved, and under the condition that the laser radar model and the environment are relatively stable, the size of each frame of data is relatively stable, the memory occupied by each FIFO can be calculated in advance, and application is performed in the initialization stage of the operation of the algorithm, so that frequent memory application is avoided, and the calculation speed is accelerated.

Of course, not all advantages described above need to be achieved at the same time in the practice of any one product or method of the present application.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below. It is to be understood that the drawings in the following description are of some embodiments of the application only. For a person skilled in the art, without inventive effort, further figures can be obtained from these figures.

Fig. 1 is a schematic flowchart illustrating a method for processing lidar data according to an embodiment of the present disclosure;

fig. 2 is an exemplary diagram of parallel processing of lidar data according to an embodiment of the present disclosure;

fig. 3 is a schematic flow chart illustrating another lidar data processing method according to an embodiment of the present disclosure;

fig. 4 is a schematic structural diagram of a laser radar data processing system according to an embodiment of the present disclosure;

fig. 5 is a block diagram illustrating a lidar data processing apparatus according to an embodiment of the present disclosure.

Detailed Description

The technical solution in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. It is to be understood that the described embodiments are merely a few embodiments of the present application and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without inventive step, are within the scope of the present disclosure.

It should be noted that the terms "comprises" and "comprising," and any variations thereof, in the examples and figures herein, are intended to cover non-exclusive inclusions. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those steps or elements listed, but may alternatively include other steps or elements not listed, or inherent to such process, method, article, or apparatus.

The application provides a laser radar data processing method and system, which can compress laser radar data from multiple angles, so that the data volume of the laser radar data is greatly reduced. The following provides a detailed description of examples of the present application.

Fig. 1 is a schematic flowchart of a laser radar data processing method according to an embodiment of the present disclosure. The method can be applied to terminal equipment such as vehicles, mobile terminals and the like, and can comprise the following steps:

step S110: the method comprises the steps of obtaining a multi-frame two-dimensional depth map converted from original data of the laser radar, and obtaining a filter determined according to the ratio of the transverse angle resolution to the longitudinal angle resolution of the laser radar.

As shown in fig. 2, a specific embodiment of acquiring a multi-frame two-dimensional depth map converted from raw lidar data may include: receiving laser radar original data sent by a laser radar; converting the laser radar original data into multi-frame laser radar point cloud data; sequentially adding each frame of laser radar point cloud data into a point cloud FIFO queue according to the conversion sequence of the laser radar original data; sequentially acquiring each frame of laser radar point cloud data from the point cloud FIFO queue according to the principle of first-in first-out of the point cloud FIFO queue, and converting each frame of laser radar point cloud data into a two-dimensional depth map; sequentially adding the converted two-dimensional depth maps into a depth map FIFO queue according to the conversion sequence of the laser radar point cloud data; and sequentially acquiring each frame of laser radar two-dimensional depth map in the depth map FIFO queue according to the principle of first-in first-out of the depth map FIFO queue.

When the laser radar original data are converted into multi-frame laser radar point cloud data, the conversion can be realized based on a laser radar driving program; when each frame of laser radar point cloud data is converted into a two-dimensional depth map, the conversion module can be used for realizing the conversion. When each frame of laser radar two-dimensional depth map in the depth map FIFO queue is obtained, intraframe and interframe compression can be obtained and realized based on the compression module, and the interframe compressed two-dimensional depth map is added into the data transmission FIFO queue, so that the integrally compressed two-dimensional depth map is transmitted to the server after the data transmission FIFO queue is integrally compressed.

Point cloud data refers to a collection of vectors in a three-dimensional coordinate system. The scanning data of the point cloud data is recorded in the form of points, each point includes three-dimensional coordinates, and some points may include color information or reflection intensity information. The maximum scanning range of the laser radar of the embodiment of the application can be 360 degrees, and other ranges can also be adopted. One frame of laser radar point cloud data corresponds to laser radar original data scanned by the laser radar in one maximum scanning range. A two-dimensional depth map (which refers to raw data of the uncompressed two-dimensional depth map) includes a plurality of data points (which may also be referred to as pixels), coordinates of each data point including depth information representing a distance between a target object characterized by the data point and the lidar and intensity information representing a lidar pulse echo intensity that generated the data point.

For each FIFO, when the number reaches the upper limit of the length, the data queued at the earliest time can be deleted, preventing memory overflow. By adopting the mechanism, the functions of the laser radar driving program, the conversion module and the compression module can be simultaneously operated through multiple threads, the frame rate of the algorithm for processing the laser radar data can be improved, the size of each frame of data is relatively stable under the condition that the type and the environment of the laser radar are relatively stable, the memory occupied by each FIFO can be calculated in advance and applied in the initialization stage of the operation of the algorithm, frequent memory application is avoided, and the calculation speed is accelerated. Meanwhile, the method can artificially regulate the length of each FIFO through the size of an application memory, when a certain link is blocked, data can be accumulated in the corresponding FIFO, the data obtained earliest can be popped up after the maximum value is reached, the whole link cannot be stopped, the robustness of a program is increased, the length of each FIFO can be monitored in real time when the program runs, and the blocked link can be quickly positioned.

In one embodiment, the specific implementation of obtaining the filter determined according to the ratio of the transverse angular resolution to the longitudinal angular resolution of the lidar includes: a filter having a height to width ratio of the transverse angular resolution to the longitudinal angular resolution of the lidar is obtained, for example, when the ratio of the transverse angular resolution to the longitudinal angular resolution is 1:5, a filter having a height of 1 and a width of 5, i.e., a size of 1 × 5, or a filter having a size of 2 × 10 may be selected. Angular resolution refers to the resolving power of an imaging system or a component of a system, i.e., the ability of an imaging system or system element to differentially distinguish between the minimum separation of two adjacent objects.

However, in practical applications, when the ratio of the transverse angular resolution and the longitudinal angular resolution of the lidar is directly used as the ratio of the height to the width of the filter for compression, the problem that the distortion rate of decompressed lidar data is high due to the fact that the compression rate is appropriate but the signal-to-noise ratio is low may occur. In order to balance the compression ratio and the signal-to-noise ratio so that both the compression ratio and the signal-to-noise ratio meet the preset compression requirement, the size of the filter can be adjusted based on the ratio of the transverse angular resolution to the longitudinal angular resolution of the laser radar. In order to improve the adjustment efficiency, before step S120 is executed, if the compression ratio and/or the signal-to-noise ratio of the filter, the ratio of which is the ratio of the transverse angular resolution to the longitudinal angular resolution of the lidar, does not meet a preset compression requirement, the size of the filter is adjusted according to the transverse distance resolution of the lidar at the distance of interest, the longitudinal distance resolution of the lidar at the distance of interest, and the size of the target object at the distance of interest, until the compression ratio and the signal-to-noise ratio of the adjusted filter meet the preset compression requirement, and a finally required filter is obtained. And the distance of interest is a distance corresponding to the raw data of the laser radar, and the size comprises height and width. The preset compression requirements comprise that the compression rate is less than or equal to a preset compression rate threshold value, and the signal-to-noise ratio is greater than or equal to a preset signal-to-noise ratio threshold value.

The compression rate is a compression rate for calculating the inter-frame compressed two-dimensional depth map obtained after the steps S120 and S130 are performed on the multi-frame two-dimensional depth map, and the calculation method includes:

where eta represents the compression ratio, c_keyRepresenting the size of the two-dimensional depth map after inter-frame compression of the first frame in a two-dimensional depth map grouping mentioned below, c_iRepresents the size of the two-dimensional depth map after inter-frame compression of the (i + 1) th frame in the two-dimensional depth map packet,

indicating the size before the intra-frame compression of the two-dimensional depth map per frame, for scenes with little environmental change,

can be approximately considered as a fixed value, n represents the number of frames in a two-dimensional depth map packet, i.e. the preset number of frames mentioned below, 0 ≦ i ≦ n-1, c₀＝c_keyAnd n is a positive integer.

In the lidar image, when two targets are located at the same azimuth but at different distances from the lidar, the minimum distance between the two targets distinguished by the lidar is the range resolution. The distance resolution calculation method comprises the following steps: calculating the transverse distance resolution of the laser radar on the distance of interest according to the distance of interest, the transverse angle resolution and the installation height of the laser radar; and calculating the longitudinal distance resolution of the laser radar on the distance of interest according to the distance of interest, the longitudinal angle resolution and the installation height of the laser radar.

The calculation process of the transverse distance resolution comprises the following steps: determining the range of interest l, lateral angular resolution a of the lidar_hengAnd substituting the installation height h into a first formula to calculate the transverse distance resolution d of the laser radar on the distance of interest_hengWherein the first formula is

And/or the calculation process of the longitudinal distance resolution comprises the following steps: determining the distance of interest l, longitudinal angular resolution a of the lidar_zongSubstituting the installation height h into a second formula to calculate the longitudinal distance resolution d of the laser radar on the distance of interest_zongWherein the second formula is

The method of resizing the filter according to the lidar transverse range resolution at a range of interest, the longitudinal range resolution at the range of interest, and the size of the target object at the range of interest includes: calculating a ratio of a height of a target object at the distance of interest to the lateral range resolution of the lidar at the distance of interest, a ratio of a width of the target object to the longitudinal range resolution, respectively; if the ratio of the height of the target object to the transverse distance resolution is larger than the ratio of the width of the target object to the longitudinal distance resolution, the size occupied by the height of the filter is increased; if the ratio of the height of the target object to the transverse distance resolution is smaller than the ratio of the width of the target object to the longitudinal distance resolution, the size occupied by the width of the filter is increased. The height is the height/(height + width), and the width is the width/(height + width).

In one embodiment, if the transverse angular resolution and the longitudinal angular resolution of the lidar are 0.1 degrees and 0.5 degrees, respectively, the distance of interest is 80m, the mounting height is 6m, and the transverse distance resolution and the longitudinal distance resolution are calculated to be 0.14m and 0.7m, respectively, then the ratio of the transverse distance resolution to the longitudinal distance resolution is 1:5, whereby the filter can be set to a size of 1 × 5 first, but when the filter based on this size is compressed, the resulting signal-to-noise ratio does not meet the preset compression requirement, so adjustment can be made based on the transverse distance resolution and the longitudinal distance resolution. Assuming that the height and width of the target object at the distance of interest are 1.5m and 5m, respectively, the ratio of the height of the target object to the lateral distance resolution is larger than the ratio of the width of the target object to the longitudinal distance resolution, whereby the size proportion of the height can be increased, finally obtaining a 2 × 6 filter.

It should be noted that, since there is no dependency relationship between "acquiring the multi-frame two-dimensional depth map converted from the raw data of the laser radar" and "acquiring the filter determined according to the transverse angular resolution and the longitudinal angular resolution of the laser radar", the execution order of the two is not limited in the present application.

Step S120: and carrying out intraframe compression on each frame of two-dimensional depth map according to the filter and a preset plane fitting algorithm to obtain the multi-frame intraframe compressed two-dimensional depth map.

Performing plane fitting on a first target area on the to-be-intra compressed two-dimensional depth map filtered by the filter according to the preset plane fitting algorithm to obtain a first fitting plane; calculating a first fitting error according to the difference between the coordinates of the plurality of data points on the first fitting plane and the coordinates of the same corresponding data point on the two-dimensional depth map to be intra-frame compressed; if the first fitting error is smaller than a preset error threshold, keeping a first fitting record, wherein the first fitting record comprises a fitting position of the first fitting plane and a fitting parameter of the first fitting plane, the fitting position of the first fitting plane is a position of the first target area on the compressed two-dimensional depth map in the frame to be compressed, and the fitting parameter of the first fitting plane comprises a normal vector and a geometric equation of the first fitting plane; if the first fitting error is larger than or equal to the preset error threshold, not keeping the first fitting record; after the filter is slid to a second target area, performing plane fitting on the first target area and the second target area as an integral area according to the preset plane fitting algorithm to obtain a second fitting plane; calculating a second fitting error according to the difference between the coordinates of the plurality of data points on the second fitting plane and the coordinates of the same corresponding data point on the to-be-intraframe compressed two-dimensional depth map before plane fitting; if the second fitting error is smaller than the preset error threshold, replacing the first fitting record with a second fitting record, wherein the second fitting record comprises a fitting position of the second fitting plane and fitting parameters of the second fitting plane, the fitting position of the second fitting plane comprises a position of the first target area and a position of the second target area, and the fitting parameters of the second fitting plane comprise a normal vector and a geometric equation of the second fitting plane; if the second fitting error is larger than or equal to the preset error threshold, performing plane fitting on the second target area again until the last target area of the to-be-intra-frame compressed two-dimensional depth map is subjected to plane fitting processing, and obtaining the to-be-intra-frame compressed two-dimensional depth map, wherein the to-be-intra-frame compressed two-dimensional depth map comprises the last reserved fitting record and original data of the target area, on which the fitting record is not reserved, of the to-be-intra-frame compressed two-dimensional depth map; and after the intraframe compression is completed on the multiple frames of two-dimensional depth maps, obtaining the multiple frames of intraframe compressed two-dimensional depth maps.

The preset plane fitting algorithm can be a linear least square method and can also be other plane fitting algorithms. The method of calculating the fitting error (including the first fitting error and the second fitting error) may include: and calculating the mean square error of the difference values of the plurality of data points according to the difference between the coordinates of the plurality of data points on the fitting plane (including the first fitting plane and the second fitting plane) and the coordinates of the corresponding same data point on the two-dimensional depth map, and taking the mean square error as the fitting error.

Assuming that the two-dimensional depth map to be intra-frame compressed can be divided into target areas with sizes of 3 filters, which are a first target area, a second target area and a third target area, respectively, if a first fitting error of a first fitting plane for the first target area is smaller than a preset error threshold, a first fitting record is reserved, which includes a fitting position of the first fitting plane (a position of the first target area) and a fitting parameter of the first fitting plane; after the filter slides to a second target area, performing plane fitting on the first target area and the second target area as an integral area to obtain a second fitting plane, and if a second fitting error of the second fitting plane is smaller than a preset error threshold, replacing a first fitting record with the second fitting record, wherein the second fitting record comprises a fitting position (positions of the first target area and the second target area) of the second fitting plane and a fitting parameter of the second fitting plane; and the filter continuously slides to a third target area, plane fitting is carried out on the first target area to the third target area as an integral area to obtain a third fitting plane, if the third fitting error of the third fitting plane is larger than or equal to a preset error threshold value, plane fitting is continuously carried out only on the third target area to obtain a fourth fitting plane, and if the fourth fitting error of the fourth fitting plane is larger than or equal to the preset error threshold value, original data of the third target area on the two-dimensional depth map are reserved. The finally obtained intra-frame compressed two-dimensional depth map comprises: the positions of the first target area and the second target area, the fitting parameters of a second fitting plane for performing plane fitting on the whole area of the first target area and the second target area, and the original data of the third target area on the two-dimensional depth map before intraframe compression.

Step S130: and for a two-dimensional depth map group, performing interframe compression on the two-dimensional depth map group according to the difference between the two-dimensional depth map compressed in the second frame to the two-dimensional depth map compressed in the last frame in the two-dimensional depth map group and the two-dimensional depth map compressed in the first frame.

Each two-dimensional depth group comprises a two-dimensional depth map which is formed by compressing continuous multi-frame frames with the number of preset frames. Before the original data of the laser radar is converted into the two-dimensional depth map and interframe compression is carried out, the two-dimensional depth map can be grouped at any time. By the above formula of compressibility

It can be known that the larger the preset frame number n is, the smaller the compression ratio is, the better the compressibility is, but the better the compressibility is, the smaller the signal-to-noise ratio is, and the higher the distortion ratio of the two-dimensional depth map is, so the value of n needs to be determined according to practical experience, so as to meet the preset compression requirement set according to the compression ratio and the signal-to-noise ratio, for example, the value may be 10.

For each frame of intra-compressed two-dimensional depth map except for a first frame of intra-compressed two-dimensional depth map in a two-dimensional depth map group, respectively calculating differences between a plurality of target areas in the current frame of intra-compressed two-dimensional depth map and target areas corresponding to the first frame of intra-compressed two-dimensional depth map, and obtaining inter-frame deviation corresponding to the current frame of intra-compressed two-dimensional depth map, wherein the target areas are the sizes of areas subjected to primary filtering by the filter, namely the target areas comprise the first target area, the second target area and the like; and reserving the compressed two-dimensional depth map in the first frame, and replacing the compressed two-dimensional depth map in the current frame with the position of the compressed two-dimensional depth map in the current frame and the corresponding inter-frame deviation.

When calculating the difference between a plurality of target areas in the compressed two-dimensional depth map in the current frame and the target area corresponding to the compressed two-dimensional depth map in the first frame, if the data contained in a target area in the compressed two-dimensional depth map in the current frame and the target area at the position corresponding to the compressed two-dimensional depth map in the first frame are fitting parameters, the difference between the two is the difference between the normal vectors of the two fitting planes and the difference between the geometric equations of the two fitting planes, if the data contained in a target area in the compressed two-dimensional depth map in the current frame and the target area at the position corresponding to the compressed two-dimensional depth map in the first frame are the original data on the two-dimensional depth map before intra-frame compression, the difference between the two is the difference between the original data (i.e. the coordinate difference of the data points), and if the data contained in a target area in the compressed two-dimensional depth map in the current frame and the position corresponding to the compressed two-dimensional depth map in the first frame are the original data on the two-dimensional depth map before intra-frame, the difference between the target area and the target area in the compressed two-frame and the first frame is the target area If the data included in the target region includes the fitting parameters and the raw data, the fitting parameters may be converted into the raw data on the two-dimensional depth map before compression in the frame, that is, the coordinates of each data point on the fitting plane are determined, and then the difference between the raw data of the two data points is calculated.

Step S140: and after all the two-dimensional depth map groups are subjected to interframe compression, all the two-dimensional depth maps subjected to interframe compression are subjected to overall compression to obtain the two-dimensional depth maps subjected to overall compression.

The entire compression may be a file compression method, or may be another compression method, for example, a file compression method such as zip or rar. After the two-dimensional depth map after the overall compression is obtained, the two-dimensional depth map after the overall compression can be stored locally, or can be sent to a server, so that the server decompresses the two-dimensional depth map and then performs operations such as statistics and analysis on the decompressed data.

When the integrally compressed two-dimensional depth map needs to be transmitted to the server, the integrally compressed two-dimensional depth map can be sequentially added to the data transmission FIFO queue according to the inter-frame compression sequence of the two-dimensional depth map grouping, and after all the inter-frame compressed two-dimensional depth groups are added to the data transmission FIFO queue, the data transmission FIFO queue is integrally compressed, so that the integrally compressed two-dimensional depth map is obtained. Specifically, the whole data transmission FIFO queue may be compressed integrally, or the data in the data transmission queue may be compressed in batches, and the integrally compressed two-dimensional depth map may be sent to the server in sequence according to the first-in first-out principle.

The processing method of the laser radar data, provided by the embodiment of the application, can obtain a multi-frame two-dimensional depth map converted from laser radar original data, obtain a filter determined according to the ratio of the transverse angle resolution to the longitudinal angle resolution of the laser radar, perform intra-frame compression on each frame of the two-dimensional depth map according to the filter and a preset plane fitting algorithm, obtain a multi-frame intra-frame compressed two-dimensional depth map, perform inter-frame compression on a two-dimensional depth map group formed by continuous multi-frame intra-frame compressed two-dimensional depth maps with the frame number being a preset frame number, and finally perform integral compression on all the two-dimensional depth maps after the inter-frame compression is completed, so as to obtain a final compression result. Therefore, the method and the device can realize intraframe compression, interframe compression and overall compression of the two-dimensional depth map converted from the original data of the laser radar, greatly reduce the data volume of the two-dimensional depth map from the aspect of multi-angle compression, and filter the two-dimensional depth map according to the filter determined by the transverse and longitudinal angle resolution, but not directly use the filter with the same width and height to filter the two-dimensional depth map, thereby improving the compression rate and the compression speed. When laser radar data (namely laser radar original data or a two-dimensional depth map) needs to be stored locally, storage space can be saved according to the embodiment of the application, and transmission efficiency can be improved when the laser radar data need to be transmitted outwards.

Based on the method embodiment, the application also provides a laser radar data processing method, and the method can be applied to terminal equipment and also can be applied to a server. When the method is applied to the terminal device, the terminal device can store the compressed two-dimensional depth map (i.e., the integrally compressed two-dimensional depth map finally obtained in fig. 1) to the local after being compressed based on the method shown in fig. 1, and when the two-dimensional depth map needs to be read, the integrally compressed two-dimensional depth map stored to the local can be determined as the two-dimensional depth map to be decompressed, and decompression is performed by adopting the embodiment of the present application; when the method is applied to a server, after the terminal device is compressed based on the method shown in fig. 1, the compressed two-dimensional depth map (i.e., the overall compressed two-dimensional depth map finally obtained in fig. 1) may be sent to the server, and the server determines the received overall compressed two-dimensional depth map as the two-dimensional depth map to be decompressed and decompresses the two-dimensional depth map by using the embodiment of the present application. As shown in fig. 3, the method includes:

step S210: and acquiring a two-dimensional depth map to be decompressed.

The two-dimensional depth map to be decompressed is data obtained by compressing a two-dimensional depth map obtained by converting original data of the laser radar, and the compression process is as shown in the embodiment shown in fig. 1, which is not described again.

Step S220: and integrally decompressing the two-dimensional depth map to be decompressed to obtain a plurality of two-dimensional depth map groups to be decompressed among frames.

The two-dimensional depth map to be inter-frame decompressed is a two-dimensional depth map group which is obtained by inter-frame compressing a continuous multi-frame intra-frame compressed two-dimensional depth map with a preset frame number. The overall decompression is an inverse algorithm of an algorithm adopted by the overall compression, for example, when the overall compression adopts a zip compression algorithm, the overall decompression adopts a zip decompression algorithm.

Step S230: and according to the difference between the two-dimensional depth map compressed in the second frame to the two-dimensional depth map compressed in the last frame in the two-dimensional depth map group to be subjected to inter-frame decompression and the two-dimensional depth map compressed in the first frame, performing inter-frame decompression on the two-dimensional depth map group to be subjected to inter-frame decompression to obtain multiple frames of two-dimensional depth maps to be subjected to intra-frame decompression.

The two-dimensional depth map to be decompressed in the frame comprises at least one fitting position, fitting parameters corresponding to the fitting position and original data of the two-dimensional depth map to be decompressed in the frame at a position which is not fitted before compression in the frame, wherein the fitting parameters comprise a normal vector of a fitting plane and a geometric equation of the fitting plane.

When the to-be-inter-frame decompressed two-dimensional depth map packet includes the two-dimensional depth map compressed in the first frame, the position of the two-dimensional depth map compressed in other frames except the two-dimensional depth map compressed in the first frame, and the inter-frame deviation of the position of the two-dimensional depth map compressed in other frames relative to the two-dimensional depth map compressed in the first frame, the specific implementation process of this step includes: for each frame of intra-frame compressed two-dimensional depth map except the first frame of intra-frame compressed two-dimensional depth map in the to-be-inter-frame decompressed two-dimensional depth map group, determining the current frame of intra-frame compressed two-dimensional depth map according to the position of the current frame of intra-frame compressed two-dimensional depth map, the inter-frame deviation of the current frame of intra-frame compressed two-dimensional depth map relative to the first frame of intra-frame compressed two-dimensional depth map, and the first frame of intra-frame compressed two-dimensional depth map; after the two-dimensional depth map compressed in each frame of the two-dimensional depth map group to be decompressed among frames is determined, the two-dimensional depth map compressed in the frames is determined to be the two-dimensional depth map to be decompressed among the frames, so that the two-dimensional depth map to be decompressed among the frames is obtained.

And determining the complete two-dimensional depth map compressed in the current frame by combining the position of the two-dimensional depth map compressed in the current frame, namely obtaining the intra-frame compressed two-dimensional depth map at which position.

Step S240: and for each frame of two-dimensional depth map to be subjected to intra-frame decompression, performing intra-frame decompression on the fitting position of the two-dimensional depth map to be subjected to intra-frame decompression according to the fitting position and the fitting parameters corresponding to the fitting position, and obtaining original data of the fitting position on the two-dimensional depth map subjected to intra-frame decompression, so that when all original data of the two-dimensional depth map to be subjected to intra-frame decompression on the two-dimensional depth map subjected to intra-frame decompression are obtained, the two-dimensional depth map subjected to intra-frame decompression is obtained.

According to the fitting position and the fitting parameter corresponding to the fitting position, intra-frame decompression is carried out on the fitting position of the two-dimensional depth map to be intra-frame decompressed, and the process of obtaining the original data of the fitting position on the intra-frame decompressed two-dimensional depth map comprises the following steps: and calculating the coordinates of each data point on the fitting plane according to the normal vector and the geometric plane of the fitting position, and obtaining the original data of the fitting position on the two-dimensional depth map after intra-frame decompression. When selecting data points on the fitting plane, the data points can be selected according to the distance between the data points on the two-dimensional depth map before intraframe compression.

According to the processing method of the laser radar data, after the two-dimensional depth map to be decompressed is obtained, overall decompression, inter-frame decompression and intra-frame decompression are sequentially carried out on the two-dimensional depth map to be decompressed, and finally the two-dimensional depth map before compression is obtained. As the compression and the decompression are reversible operation, the decompression process shows that the intraframe compression, the interframe compression and the integral compression can be sequentially carried out on the two-dimensional depth map converted from the original data of the laser radar, and the data volume of the two-dimensional depth map is greatly reduced from the multi-angle compression aspect.

Corresponding to the above method embodiment, an embodiment of the present application provides a system for processing laser radar data, where as shown in fig. 4, the system includes a laser radar, a terminal device, and a server;

the terminal device is configured to obtain an overall compressed two-dimensional depth map by executing the method in the embodiment corresponding to fig. 1, and send the overall compressed two-dimensional depth map to the server; wherein the compression process comprises intraframe compression, interframe compression and overall compression;

Based on the foregoing method embodiment, another embodiment of the present application provides an apparatus for processing lidar data, as shown in fig. 5, the apparatus includes:

a depth map acquisition unit 30 configured to acquire a multi-frame two-dimensional depth map into which raw data of the laser radar is converted;

a filter obtaining unit 32 configured to obtain a filter determined according to a ratio of a transverse angular resolution to a longitudinal angular resolution of the laser radar;

the intraframe compression unit 34 is configured to perform intraframe compression on each frame of two-dimensional depth map according to the filter and a preset plane fitting algorithm, and obtain a plurality of frames of intraframe compressed two-dimensional depth maps;

an inter-frame compression unit 36, configured to, for a two-dimensional depth map packet, inter-frame compress the two-dimensional depth map packet according to a difference between a two-dimensional depth map compressed in a second frame to a two-dimensional depth map compressed in a last frame in the two-dimensional depth map packet and a two-dimensional depth map compressed in a first frame, where each two-dimensional depth packet includes two-dimensional depth maps compressed in consecutive frames, and the number of frames of each two-dimensional depth packet is a preset number of frames;

and an overall compression unit 38, configured to perform inter-frame compression on all the two-dimensional depth map groups, and then perform overall compression on all the two-dimensional depth maps after the inter-frame compression, so as to obtain an overall compressed two-dimensional depth map.

In one embodiment, the depth map acquiring unit 30 is configured to acquire a filter having a ratio of height to width that is a ratio of a lateral angular resolution to a longitudinal angular resolution of the lidar;

the device further comprises:

In one embodiment, the adjusting unit includes:

In one embodiment, the intra-frame compression unit 34 includes:

In one embodiment, the inter-frame compression unit 36 includes:

In one embodiment, the depth map obtaining unit 30 includes:

In one embodiment, the integral compression unit 38 includes:

In one embodiment, the apparatus comprises:

In one embodiment, an inter-frame decompression unit includes:

Based on the above method embodiments, another embodiment of the present application provides a storage medium having stored thereon executable instructions that, when executed by a processor, cause the processor to implement the method as described above.

Based on the foregoing method embodiment, another embodiment of the present application provides an electronic device, including:

one or more processors;

a storage device for storing one or more programs,

wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the method as described above.

The system and apparatus embodiments correspond to the method embodiments, and have the same technical effects as the method embodiments, and for the specific description, refer to the method embodiments. The device embodiment is obtained based on the method embodiment, and for specific description, reference may be made to the method embodiment section, which is not described herein again. Those of ordinary skill in the art will understand that: the figures are merely schematic representations of one embodiment, and the blocks or processes in the figures are not necessarily required to practice the present application.

Those of ordinary skill in the art will understand that: modules in the devices in the embodiments may be distributed in the devices in the embodiments according to the description of the embodiments, or may be located in one or more devices different from the embodiments with corresponding changes. The modules of the above embodiments may be combined into one module, or further split into multiple sub-modules.

Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions in the embodiments of the present application.

Claims

1. A method of lidar data processing, the method comprising:

2. The method of claim 1, wherein obtaining a filter determined from a ratio of a transverse angular resolution and a longitudinal angular resolution of the lidar comprises: obtaining a filter with the height-width ratio being the ratio of the transverse angular resolution to the longitudinal angular resolution of the laser radar;

3. The method of claim 2, wherein adjusting the size of the filter according to the lidar transverse range resolution at a range of interest, the longitudinal range resolution at the range of interest, and the size of the target object at the range of interest comprises:

4. The method of claim 1, wherein intra-frame compressing each frame of two-dimensional depth map according to the filter and a preset plane fitting algorithm to obtain a plurality of frames of intra-frame compressed two-dimensional depth maps comprises:

5. The method of claim 1, wherein for a two-dimensional depth map packet, inter-frame compressing the two-dimensional depth map packet according to differences between a two-dimensional depth map compressed in a second frame to a two-dimensional depth map compressed in a last frame in the two-dimensional depth map packet and a two-dimensional depth map compressed in a first frame, respectively, comprises:

6. The method of claim 1, wherein obtaining the multi-frame two-dimensional depth map into which the raw lidar data is converted comprises:

receiving laser radar original data sent by the laser radar;

7. The method of claim 6, wherein after performing inter-frame compression on all the two-dimensional depth map groups, performing overall compression on all the inter-compressed two-dimensional depth maps to obtain an overall compressed two-dimensional depth map, and including:

8. The method according to any of claims 1-7, wherein after performing an overall compression on all the inter-compressed two-dimensional depth maps to obtain an overall compressed two-dimensional depth map, the method comprises:

9. The method according to claim 8, wherein when the two-dimensional depth map packet to be inter-frame decompressed includes inter-frame deviations of the two-dimensional depth map packet to be inter-frame decompressed from the first frame intra-frame compressed two-dimensional depth map, the positions of the two-dimensional depth maps in other frames except the first frame intra-frame compressed two-dimensional depth map, and the positions of the two-dimensional depth maps in other frames relative to the two-dimensional depth map packet to be intra-frame compressed in the first frame intra-frame, the two-dimensional depth map packet to be inter-frame decompressed is inter-frame decompressed according to differences between the two-dimensional depth maps in the second frame intra-frame compressed two-dimensional depth map to the last frame intra-frame compressed two-dimensional depth map in the two-dimensional depth map packet to be inter-frame decompressed and the two-dimensional depth map compressed in the first frame intra-frame, so as to obtain a multi-frame intra-frame decompressed two-dimensional depth map, the method comprises the following steps:

10. The system for processing the laser radar data is characterized by comprising a laser radar, terminal equipment and a server;

the terminal device is used for obtaining an overall compressed two-dimensional depth map by executing the method of any one of claims 1-7, and sending the overall compressed two-dimensional depth map to the server;