WO2022017131A1 - 点云数据的处理方法、智能行驶控制方法及装置 - Google Patents

点云数据的处理方法、智能行驶控制方法及装置 Download PDF

Info

Publication number
WO2022017131A1
WO2022017131A1 PCT/CN2021/102706 CN2021102706W WO2022017131A1 WO 2022017131 A1 WO2022017131 A1 WO 2022017131A1 CN 2021102706 W CN2021102706 W CN 2021102706W WO 2022017131 A1 WO2022017131 A1 WO 2022017131A1
Authority
WO
WIPO (PCT)
Prior art keywords
sparse matrix
point cloud
target
layer
cloud data
Prior art date
Application number
PCT/CN2021/102706
Other languages
English (en)
French (fr)
Inventor
王哲
宋潇
杨国润
石建萍
Original Assignee
商汤集团有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 商汤集团有限公司 filed Critical 商汤集团有限公司
Publication of WO2022017131A1 publication Critical patent/WO2022017131A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02—Control of position or course in two dimensions
    • G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0231—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means
    • G05D1/0246—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using a video camera in combination with image processing means
    • G05D1/0251—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using a video camera in combination with image processing means extracting 3D information from a plurality of images taken from different locations, e.g. stereo vision
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/88—Lidar systems specially adapted for specific applications
    • G01S17/93—Lidar systems specially adapted for specific applications for anti-collision purposes
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/88—Lidar systems specially adapted for specific applications
    • G01S17/93—Lidar systems specially adapted for specific applications for anti-collision purposes
    • G01S17/931—Lidar systems specially adapted for specific applications for anti-collision purposes of land vehicles
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/88—Lidar systems specially adapted for specific applications
    • G01S17/93—Lidar systems specially adapted for specific applications for anti-collision purposes
    • G01S17/933—Lidar systems specially adapted for specific applications for anti-collision purposes of aircraft or spacecraft
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
    • G01S7/48—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
    • G01S7/48—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00
    • G01S7/4802—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00 using analysis of echo signal for target characterisation; Target signature; Target cross-section
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02—Control of position or course in two dimensions
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02—Control of position or course in two dimensions
    • G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0212—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory
    • G05D1/0221—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory involving a learning process
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02—Control of position or course in two dimensions
    • G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0257—Control of position or course in two dimensions specially adapted to land vehicles using a radar
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02—Control of position or course in two dimensions
    • G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0276—Control of position or course in two dimensions specially adapted to land vehicles using signals provided by a source external to the vehicle

Definitions

  • the present disclosure relates to the technical field of data processing, and in particular, to a method for processing point cloud data, a method and device for intelligent driving control.
  • LiDAR is widely used in autonomous driving, UAV exploration, map mapping and other fields.
  • various applications such as target detection, mapping, localization, point cloud segmentation, and scene flow have been generated.
  • target detection, mapping, localization, point cloud segmentation, and scene flow have been generated.
  • a large amount of sample data with different distributions is usually used to train the model; All point cloud data must have good processing ability, which leads to a significant increase in the difficulty of algorithm design.
  • the embodiments of the present disclosure provide at least a point cloud data processing method, an intelligent driving control method, an apparatus, a computer device, and a storage medium.
  • an embodiment of the present disclosure provides a method for processing point cloud data, including: acquiring first point cloud data obtained by a target radar collecting a target scene in a first installation attitude; and based on a predetermined first transformation matrix, Converting the first point cloud data into second point cloud data in a preset standard attitude; performing a detection task on the second point cloud data to obtain a target detection result.
  • the first point cloud data of the target scene collected by the target radar in the first installation attitude is converted into the second point cloud data under the preset standard attitude, and the second point cloud data is
  • the cloud data performs the detection task and obtains the target detection result, so that the point cloud data in different installation postures can be converted to the standard installation posture to perform the detection task, so that the algorithm for performing the detection task only needs to perform the point cloud data in the standard installation posture. It is enough to have good processing ability to reduce the difficulty of algorithm design.
  • the first installation posture satisfies at least one of the following: a first installation angle of the target radar in the first installation posture and an angle of the target radar in the preset standard posture.
  • the angle difference between the standard angles is less than a preset angle difference threshold; the distance between the detection center of the target radar in the first installation attitude and the detection center of the target radar in the preset standard attitude less than the preset distance threshold.
  • the first installation attitude of the target radar is limited to the vicinity of the preset standard attitude, so that the feature distribution of the first point cloud data obtained under the first installation attitude and the characteristics of the point cloud data obtained under the preset standard attitude The distribution is close, so that the installation posture of the target radar will not be excessively restricted, and the accuracy of the target detection result is guaranteed.
  • At least one of the angle difference threshold and the distance threshold is determined in the following manner: the angle difference threshold is determined based on detection results obtained by performing detection on multiple verification sample sets respectively. and at least one of the distance thresholds, wherein each verification sample set includes multiple sets of verification data; the radars used for collecting different verification sample sets have different installation attitudes.
  • At least one of the angular difference threshold and the distance threshold is determined, so that at least one of the obtained angular difference threshold and the distance threshold can have higher accuracy, ensuring the accuracy of the target detection result.
  • the first transformation matrix is determined in the following manner: based on the actual detection data of the target radar in the first installation attitude, and the predetermined position of the target radar in the predetermined position. Assuming standard detection data in a standard attitude, the first transformation matrix is determined.
  • the performing a detection task on the second point cloud data to obtain a target detection result includes: generating at least one corresponding to the second point cloud data based on the second point cloud data. a sparse matrix; the sparse matrix is used to represent whether there are target objects at different positions of the target scene; based on the at least one sparse matrix and the second point cloud data, determine the target objects included in the target scene the three-dimensional detection data; use the three-dimensional detection data as the target detection result.
  • At least one corresponding sparse matrix can be generated for the acquired second point cloud data, and the sparse matrix is used to represent whether there are target objects at different positions of the target scene; in this way, based on the sparse matrix and the second point cloud data,
  • the target position of the corresponding target object can be determined based on the sparse matrix, so that the features corresponding to the target position can be processed, and other positions in different positions except the target position can be corresponding to the target position.
  • the features of the target object are not processed, which reduces the amount of calculation to obtain the three-dimensional detection data of the target object and improves the detection efficiency.
  • generating at least one sparse matrix corresponding to the second point cloud data based on the second point cloud data includes: determining, based on the second point cloud data, a method for detecting the target.
  • the sparse matrix corresponding to each layer of convolution modules in the object's neural network includes: determining, based on the second point cloud data, a method for detecting the target.
  • a corresponding sparse matrix can be determined for each layer of the convolution module of the neural network, so that each layer of the convolution module can process the input feature map based on the sparse matrix.
  • determining the sparse matrix corresponding to each layer of convolution modules in the neural network based on the second point cloud data includes: generating an initial sparse matrix based on the second point cloud data ; based on the initial sparse matrix, determine a sparse matrix that matches the target size of the feature map input to the convolution module of each layer of the neural network.
  • an initial sparse matrix can be generated based on the second point cloud data, and then based on the initial sparse matrix, a corresponding sparse matrix can be determined for each layer of the convolution module of the neural network, and the corresponding sparse matrix of each layer of the convolution module is the same as the input
  • the target size of the feature map to the convolution module of this layer is matched, so that each layer of the convolution module can process the input feature map based on the sparse matrix.
  • generating an initial sparse matrix based on the second point cloud data includes: determining a target area corresponding to the second point cloud data, and dividing the target area according to a preset number of grids. The area is divided into multiple grid areas; based on the grid area where the point corresponding to the second point cloud data is located, the matrix element value corresponding to each grid area is determined; based on the matrix element value corresponding to each grid area , and generate an initial sparse matrix corresponding to the second point cloud data.
  • the matrix element value of each grid area can be determined. If there is a point corresponding to the second point cloud data, the value of the matrix element of the grid area is 1, indicating that there is a target object at the position of the grid area, and then based on the value of the matrix element corresponding to each grid area, an initial sparse matrix is generated. , which provides data support for the subsequent determination of the 3D detection data of the target object.
  • a sparse matrix matching the target size of the feature map input to the convolution module of each layer of the neural network including any of the following: based on the initial sparse matrix Sparse matrix, determine the output sparse matrix corresponding to each layer of convolution module in the neural network, and use the output sparse matrix as the sparse matrix corresponding to the convolution module of this layer; based on the initial sparse matrix, determine the In the neural network, the input sparse matrix corresponding to the convolution module of each layer is used as the sparse matrix corresponding to the convolution module of this layer; based on the initial sparse matrix, it is determined that in the neural network, each The input sparse matrix and the output sparse matrix corresponding to the layer convolution module are fused, and the fused sparse matrix is obtained by fusing the input sparse matrix and the output sparse matrix, and the fused sparse matrix is used as the sparse matrix corresponding
  • the sparse matrix can be the input sparse matrix, the output sparse matrix, or the fusion sparse matrix generated based on the input sparse matrix and the output sparse matrix. matrix.
  • determining the input sparse matrix corresponding to each layer of convolution module in the neural network includes: using the initial sparse matrix as the first sparse matrix of the neural network.
  • the input sparse matrix corresponding to the layer convolution module based on the input sparse matrix corresponding to the i-1 layer convolution module, determine the feature map corresponding to the i layer convolution module and input to the i layer convolution module.
  • the input sparse matrix matching the target size; wherein, i is a positive integer greater than 1 and less than n+1, and n is the total number of layers of the convolution module of the neural network.
  • the initial sparse matrix can be used as the input sparse matrix corresponding to the convolution module of the first layer, and the input sparse matrix of each layer of convolution module can be determined in turn, and then the sparse matrix can be determined based on the input sparse matrix, and the subsequent sparse matrix can be determined based on the input sparse matrix.
  • a sparse matrix of one-layer convolutional modules provides data support for determining the 3D detection data of the target object.
  • the determining, based on the initial sparse matrix, the output sparse matrix corresponding to each layer of convolution modules in the neural network includes: based on the size threshold of the target object and the initial sparse matrix. sparse matrix, determine the output sparse matrix corresponding to the neural network; based on the output sparse matrix, generate a target size corresponding to the nth layer convolution module that matches the target size of the feature map input to the nth layer convolution module Output sparse matrix; based on the output sparse matrix corresponding to the j+1th layer convolution module, generate the output sparse matrix corresponding to the jth layer convolution module and matching the target size of the feature map input to the jth layer convolution module matrix, where j is a positive integer greater than or equal to 1 and less than n, and n is the total number of layers of convolution modules of the neural network.
  • the output sparse matrix can be determined based on the initial sparse matrix, and the output sparse matrix of the n-th layer convolution module, .
  • the output sparse matrix determines the sparse matrix, which provides data support for the subsequent determination of the 3D detection data of the target object based on the sparse matrix of each layer of convolution modules.
  • determining the three-dimensional detection data of the target object included in the target scene based on the at least one sparse matrix and the second point cloud data includes: based on the second point cloud data , generate a target point cloud feature map corresponding to the second point cloud data; based on the target point cloud feature map and the at least one sparse matrix, use a neural network for detecting target objects to determine the target included in the target scene.
  • generating a target point cloud feature map corresponding to the second point cloud data based on the second point cloud data includes: for each grid area, The coordinate information of the point indicated by the second point cloud data is determined, and the feature information corresponding to the grid area is determined; wherein, the grid area is the second point cloud according to the preset number of grids.
  • the target area corresponding to the data is divided and generated; based on the feature information corresponding to each grid area, the target point cloud feature map corresponding to the second point cloud data is generated.
  • a target point cloud feature map corresponding to the second point cloud data is generated, and the target point cloud feature map includes the position information of each point, and then based on the target point cloud feature map and At least one sparse matrix can more accurately determine the three-dimensional detection data of the target object included in the target scene.
  • using a neural network for detecting target objects to determine the three-dimensional detection data of the target objects included in the target scene including: based on: The sparse matrix corresponding to the first-layer convolution module in the neural network determines the feature information to be convolved in the target point cloud feature map, and uses the first-layer convolution module to analyze the target point cloud feature map.
  • the feature information to be convolved in is subjected to convolution processing to generate a feature map input to the second layer convolution module; based on the sparse matrix corresponding to the kth layer convolution module in the neural network, it is determined that the input to the
  • the feature information to be convoluted in the feature map of the k-layer convolution module is used to convolve the feature information to be convolved in the feature map of the k-th layer convolution module by using the k-th layer convolution module of the neural network.
  • the sparse matrix corresponding to the n-th layer convolution module in the network determines the feature information to be convolved in the feature map input to the n-th layer convolution module, and uses the n-th layer convolution module of the neural network.
  • the feature information to be convoluted in the feature map of the n-th layer convolution module is subjected to convolution processing to obtain three-dimensional detection data of the target object included in the target scene.
  • the feature information to be convoluted can be determined, the feature information to be convoluted can be convolved, and other features in the feature map except the feature information to be convolved can be processed.
  • the feature information is not subjected to convolution processing, which reduces the calculation amount of convolution processing performed by each layer of convolution module, improves the operation efficiency of each layer of convolution module, which can reduce the computational load of neural network and improve the detection of target objects. efficient.
  • a neural network for detecting target objects is used to determine the three-dimensional detection data of the target objects included in the target scene, including: For each convolution module of the neural network except the last layer of convolution module, the convolution module of this layer is determined based on the sparse matrix corresponding to the convolution module of this layer and the feature map input to the convolution module of this layer.
  • the convolution vector corresponding to the module based on the convolution vector corresponding to the convolution module of this layer, determine the feature map input to the convolution module of the next layer; based on the sparse matrix corresponding to the convolution module of the last layer and input to the last layer
  • the feature map of the convolution module of the last layer is used to determine the convolution vector corresponding to the convolution module of the last layer; based on the convolution vector corresponding to the convolution module of the last layer, the three-dimensional detection data of the target object included in the target scene is determined. .
  • a convolution vector corresponding to each layer of convolution module can be generated, and the convolution vector includes the feature information to be processed in the feature map.
  • the processed feature information is: the feature information in the feature map that matches the position of the three-dimensional detection data of the target object indicated in the sparse matrix, the generated convolution vector is processed, and the features to be processed are removed from the feature map.
  • Other feature information other than the information is not processed, which reduces the calculation amount of convolution processing of each layer of convolution module, improves the operation efficiency of each layer of convolution module, which can reduce the calculation amount of the neural network and improve the target. Object detection efficiency.
  • an embodiment of the present disclosure provides an intelligent driving control method, including: collecting point cloud data by using a target radar set on a driving device; The point cloud data is detected and processed to obtain a detection result; based on the detection result, the driving device is controlled.
  • the point cloud data collected by the target radar after acquiring the point cloud data collected by the target radar, the point cloud data collected by the target radar will be converted to the marked installation posture for detection processing, and the detection result will be obtained, so as to perform the detection task.
  • the algorithm only needs to have good processing ability for the point cloud data in the standard installation posture, which reduces the difficulty of the algorithm design.
  • an embodiment of the present disclosure provides a point cloud data processing device, including: an acquisition module for acquiring first point cloud data obtained by a target radar collecting a target scene in a first installation attitude; a conversion module for Based on a predetermined first transformation matrix, the first point cloud data is converted into second point cloud data in a preset standard attitude; a detection module is used to perform a detection task on the second point cloud data to obtain a target Test results.
  • an embodiment of the present disclosure further provides an intelligent driving control device, including: an acquisition module for acquiring point cloud data collected by a target radar set on the driving device; and a processing module for based on any one of the first aspect
  • the point cloud data processing method described in item 1 performs detection processing on the point cloud data to obtain a detection result; a control module is configured to control the traveling device based on the detection result.
  • an optional implementation manner of the present disclosure further provides a computer device, including a processor and a memory, where the memory stores machine-readable instructions executable by the processor, and the machine-readable instructions are processed by the processor When executed by the processor, when the machine-readable instructions are executed by the processor, the above-mentioned first aspect or the steps in any possible implementation manner of the first aspect are performed; or the steps in the implementation manners of the foregoing second aspect are performed. step.
  • an optional implementation manner of the present disclosure further provides a computer-readable storage medium, on which a computer program is run to execute the steps in the first aspect or any possible implementation manner of the first aspect ; or perform the steps in the embodiments of the second aspect above.
  • FIG. 1 shows a flowchart of a method for processing point cloud data provided by an embodiment of the present disclosure
  • FIG. 2 shows a flowchart of a specific method for performing a detection task on second point cloud data provided by an embodiment of the present disclosure to obtain a target detection result
  • Fig. 3 shows the flow chart of the specific method for determining the sparse matrix corresponding to each layer of convolution module in the neural network based on the second point cloud data provided by the embodiment of the present disclosure
  • FIG. 4 shows a schematic diagram of a target area and an initial sparse matrix corresponding to the target area provided by an embodiment of the present disclosure
  • FIG. 5 shows a flowchart of an intelligent driving control method provided by an embodiment of the present disclosure
  • FIG. 6 shows a schematic diagram of an apparatus for processing point cloud data provided by an embodiment of the present disclosure
  • FIG. 7 shows a schematic diagram of an intelligent driving control device provided by an embodiment of the present disclosure
  • FIG. 8 shows a schematic diagram of a computer device provided by an embodiment of the present disclosure.
  • lidars with different installation attitudes are generally used to obtain sample data, and the model is trained based on the sample data obtained by multiple lidars, so that the neural network model can fully learn to the corresponding features of the 3D point cloud data with different feature distributions.
  • the more features a neural network model needs to learn the more complex the corresponding algorithm will be, resulting in a significant increase in the difficulty of algorithm design.
  • the present disclosure provides a method and device for processing point cloud data, using a predetermined first transformation matrix to convert the first point cloud data of the target scene collected by the target radar in the first installation attitude into standard
  • the second point cloud data in the posture is obtained, and the detection task is performed on the second point cloud data to obtain the target detection result, so that the point cloud data in different installation postures can be converted to the standard installation posture to perform the detection task, so as to perform the detection.
  • the algorithm of the task only needs to have good processing ability for the point cloud data in the standard installation posture, which reduces the difficulty of algorithm design.
  • the execution subject of the method for processing point cloud data provided by the embodiment of the present disclosure is generally a data processing device.
  • the data processing device includes, for example, a terminal device or a server or other processing device, and the terminal device can be a user equipment (User Equipment, UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (Personal Digital Assistant) , PDA), handheld devices, computing devices, in-vehicle devices, wearable devices, etc.
  • the method for processing point cloud data may be implemented by the processor calling computer-readable instructions stored in the memory.
  • the method for processing point cloud data provided by the embodiments of the present disclosure can be applied to multiple fields, such as the field of intelligent driving, intelligent robots, unmanned aerial vehicles, augmented reality (AR), virtual reality (VR), Ocean exploration, 3D printing and other fields.
  • the following takes the application field of intelligent driving as an example to describe the processing method of the point cloud data provided by the embodiment of the present disclosure.
  • an embodiment of the present disclosure provides a method for processing point cloud data, the method includes steps S101 to S103, wherein:
  • S101 Acquire first point cloud data obtained by the target radar collecting the target scene in the first installation attitude.
  • S103 Perform a detection task on the second point cloud data to obtain a target detection result.
  • the embodiment of the present disclosure uses a predetermined first transformation matrix to convert the first point cloud data of the target scene collected by the target radar in the first installation attitude into the second point cloud data in the standard attitude, and analyzes the second point cloud data.
  • the data performs the detection task and obtains the target detection result, so that the point cloud data in different installation postures can be converted to the standard installation posture to perform the detection task, so that the algorithm for performing the detection task only needs to have the point cloud data in the standard installation posture.
  • Good processing ability is enough to reduce the difficulty of algorithm design.
  • the neural network structure determined based on the algorithm is also relatively simple, and the volume of the neural network is correspondingly smaller, which is more suitable for deployment in embedded devices.
  • a neural network with a simple algorithm needs to consume less computing resources when performing task processing, thereby reducing hardware costs.
  • the target radar is wired or wirelessly connected to the data processing device.
  • the data processing device can perform subsequent processing on the first point cloud data of the target radar in its first installation attitude.
  • the first installation posture of the target radar refers to the installation posture of the target radar when it is actually installed in the intelligent driving device.
  • the preset standard attitude refers to the reference installation attitude in the intelligent driving device determined for the target radar. For example, when the detection center of the target radar is located at position A in the intelligent driving device, and the detection axis of the target radar coincides with a preset axis L, the target radar is in a preset standard attitude.
  • the reference coordinate system is established with the A position as the origin.
  • the z-axis of the reference coordinate system is, for example, the axis L, and the spatial point where the A position is located passes through the axis L; the plane where the x-axis and the y-axis are located is perpendicular to the z-axis, and the point where the A position is located is located on the plane.
  • the radar coordinate system is established based on the first installation attitude of the target radar in the actual installation.
  • the origin of the radar coordinate system is the detection center of the radar
  • the z-axis is the detection axis of the radar
  • the plane where the x-axis and the y-axis are located is perpendicular to the z-axis
  • the point where the B position is located is located on the plane where the x-axis and the y-axis are located.
  • the target radar Since there is a difference between the actual first installation attitude of the target radar and the preset standard attitude, the target radar has a certain angle deviation and a certain distance displacement relative to the preset standard attitude.
  • the horizontal direction is used as the preset installation reference
  • the detection center of the target radar at the preset installation position is used as the origin
  • the straight line in the horizontal direction is used as the z-axis
  • the plane perpendicular to the z-axis is used as x.
  • the first installation attitude of the target radar can be determined, for example, by the angle between the detection axis of the target radar and the x, y and z axes, and the detection center of the target radar relative to.
  • the displacement of the origin is represented, for example, it can be expressed as: ( ⁇ , ⁇ , x 0 , y 0 , z 0 ).
  • ⁇ represents the angle between the detection axis and the x-axis of the reference coordinate system; represents the angle between the detection axis and the y-axis of the reference coordinate system, ⁇ represents the angle between the detection axis and the z-axis of the reference coordinate system, x 0 , y 0 , z 0 represent the target radar in the first installation attitude
  • the coordinate value of the detection center in the reference coordinate system When , the coordinate value of the detection center in the reference coordinate system.
  • the first installation posture satisfies the following At least one of: the first angle difference between the installation angle corresponding to the first installation attitude and the standard angle corresponding to the preset standard attitude is smaller than the preset angle difference threshold; the detection center corresponding to the first installation attitude is the same as the preset angle difference threshold; The distance between the detection centers corresponding to the standard posture is less than a preset distance threshold.
  • the detection center of the target radar and the center of the reference coordinate system are completely coincident in the first installation attitude of the actual installation.
  • the detection center corresponding to the first installation attitude is the same as the preset standard.
  • the distance between the detection centers corresponding to the pose is equal to 0.
  • the angle difference threshold can be expressed as: ( ⁇ , ⁇ )
  • the distance threshold can be expressed as: ( ⁇ x, ⁇ y, ⁇ z), for example.
  • the neural network when the neural network is used to perform the detection task on the second point cloud data, the first installation posture of the target radar is limited to a certain extent, thereby ensuring the feature distribution of the second point cloud data and the training samples for training the neural network. It is as close as possible to ensure that the features that the neural network needs to learn will not be too many, reduce the design difficulty of the neural network and the complexity of the trained neural network, and make the resulting neural network smaller and more suitable for deployment in embedded In the type of equipment, the neural network needs less computing resources when performing detection tasks, reducing hardware costs.
  • An embodiment of the present disclosure provides a specific method for determining at least one of an angle difference threshold and a distance threshold, including: determining the angle difference threshold and the distance threshold based on detection results obtained by detecting multiple verification sample sets respectively At least one of, wherein each verification sample set includes multiple sets of verification data; the radars used for collecting different verification sample sets have different installation attitudes.
  • a neural network can be used to perform detection tasks on multiple verification sample sets, respectively, to obtain detection results corresponding to the multiple verification sample sets; for each verification sample set in the multiple verification sample sets, each verification sample set The set includes multiple sets of verification data; the multiple sets of verification data in the same verification sample set come from the same verification radar.
  • the installation attitude of the verification radar can be randomly determined, and then multiple sets of verification data can be obtained by using the verification radar under the installation attitude. Then, at least one of an angle difference threshold and a distance threshold is determined using multiple sets of verification data obtained by the verification radar at different installation attitudes.
  • the verification radars that obtain different verification data sets can be the same radar or different radars.
  • the same type of radar can be used to obtain training samples, and correspondingly, the same type of radar can be used to obtain the verification sample set.
  • the corresponding detection results can be determined according to the respective detection results of the multiple sets of verification data in each verification sample set of the neural network.
  • the detection loss corresponding to each verification sample set, and then the angle difference threshold is determined based on the detection losses corresponding to the multiple verification sample sets respectively.
  • the angle difference threshold as an example, for example, from multiple verification sample sets, a plurality of verification sample sets whose detection loss is less than a certain loss threshold can be determined, and the second angle difference corresponding to each of the determined multiple verification sample sets can be determined.
  • the maximum value in is determined as the angle difference threshold; or, the average value of the second angle differences corresponding to the determined multiple verification sample sets respectively is used as the angle difference threshold.
  • multiple verification intervals may also be predetermined, and the i-th verification interval may be expressed as: and / or
  • Still determining the angle difference as an example, obtain multiple sets of verification samples under each verification interval, and then use the pre-trained neural network to perform detection tasks on the multiple sets of verification sample sets under each verification interval.
  • the detection results corresponding to the multiple verification sample sets under the verification interval are determined, the detection loss corresponding to each verification interval is determined, and the angle difference threshold is determined based on the verification loss corresponding to each verification interval.
  • the manner of determining the distance threshold is similar to the manner of determining the angle difference threshold, and details are not described herein again.
  • the first transformation matrix is a transformation matrix for transforming the first point cloud data from the first installation posture to the preset standard posture.
  • the radar coordinate system is established based on the first installation position of the target radar, and in the first point cloud data of the target radar in the first installation attitude, any point of the point cloud is in the radar coordinate system established based on the target radar.
  • the coordinate value is: (x, y, z)
  • the coordinate value of the point in the reference coordinate system is (x', y', z')
  • (x, y, z) and (x', y', z′) satisfy the following formula (1):
  • r ij is a rotation parameter
  • t k is a translation parameter
  • the following method can be adopted: based on the actual detection data of the target radar in the first installation attitude and the predetermined standard of the target radar in the preset standard attitude The data is detected, and the first transformation matrix is determined.
  • the calibration process of the target radar can be regarded as a process of solving the first transformation matrix based on the actual detection data and the standard detection data.
  • the actual detection data includes the first position information of the multiple position points in the detection data obtained by the target radar respectively, that is, the above (x, y, z), and the multiple position points are pre-marked in the reference coordinate system
  • the second position information of namely (x', y', z')
  • the first transformation matrix is jointly solved, and the target is finally obtained.
  • the first transformation matrix of the radar in the first installation attitude is regarded as a process of solving the first transformation matrix based on the actual detection data and the standard detection data.
  • the first point cloud data of the target radar in its first installation attitude can be converted and processed based on the transformation matrix and the above formula (1) to obtain the second point cloud data in the preset standard attitude. point cloud data.
  • a neural network when performing the detection task on the second point cloud data, for example, a neural network may be used to perform the detection task on the second point cloud data.
  • the neural network is obtained by training the training samples under the preset standard posture, wherein the training samples under the preset standard posture can be obtained, for example, in the following manner:
  • the second transformation matrix of the sample radar is obtained in a manner similar to that of the first transformation matrix of the target radar, which is not repeated here.
  • the second installation posture satisfies at least one of the following: the second angle difference between the second installation angle corresponding to the second installation posture and the standard angle corresponding to the preset standard posture is less than a preset angle difference threshold; The second distance between the detection center corresponding to the second installation posture and the detection center corresponding to the preset standard posture is smaller than a preset distance threshold.
  • the intelligent driving device is controlled to drive a certain distance, and the third point cloud data is collected during the driving process of the intelligent driving device; and then based on the second transformation matrix, the third point cloud data is converted into Transform into fourth point cloud data in a preset standard attitude, and train the neural network to be trained based on the fourth point cloud data.
  • the fourth point cloud data can also be marked to obtain the fourth point cloud data. Labeling information of cloud data; the labeling information is used to obtain the loss of the neural network in the process of training the neural network to be trained, and then adjust the parameters of the neural network to be trained based on the loss. After multiple rounds of adjustment of the parameters of the neural network to be trained, a pre-trained neural network is obtained.
  • the third point cloud data is marked to obtain the marking information of the third point cloud data
  • the third point cloud data is converted into the fourth point cloud data
  • the label information of the third point cloud data is converted into label information adapted to the fourth point cloud data.
  • another specific method for performing a detection task on second point cloud data to obtain a target detection result including:
  • S201 Based on the second point cloud data, generate at least one sparse matrix corresponding to the second point cloud data; the sparse matrix is used to represent whether there are target objects at different positions of the target scene;
  • S202 Based on the at least one sparse matrix and the second point cloud data, determine the three-dimensional detection data of the target object included in the target scene;
  • At least one corresponding sparse matrix may be generated for the acquired second point cloud data, and the sparse matrix is used to represent whether there are target objects at different positions of the target scene; in this way, based on the sparse matrix and the second Point cloud data, when determining the three-dimensional detection data of the target object, the target position of the corresponding target object can be determined based on the sparse matrix, so that the features corresponding to the target position can be processed, and the different positions are divided by the target position. The features corresponding to other positions are not processed, which reduces the amount of calculation to obtain the three-dimensional detection data of the target object and improves the detection efficiency.
  • At least one sparse matrix corresponding to the second point cloud data may be generated based on the second point cloud data.
  • the sparse matrix can represent whether there are target objects at different positions of the target scene.
  • the sparse matrix may be a matrix including 0 and 1, that is, the sparse matrix is obtained by using 0 or 1 as a matrix element value.
  • the value of the matrix element corresponding to the position where the target object exists in the target scene may be set to 1
  • the value of the matrix element corresponding to the position where the target object does not exist in the target scene may be set to 0.
  • generating at least one sparse matrix corresponding to the second point cloud data based on the second point cloud data may include: determining, based on the second point cloud data, that in the neural network for detecting the target object, each A sparse matrix corresponding to a layer of convolution modules.
  • the neural network may include multiple layers of convolution modules, and each layer of convolution modules may include one layer of convolution layers.
  • a corresponding sparse matrix may be determined for each layer of convolution modules, that is, the volume of each layer
  • Each of the product layers determines the corresponding sparse matrix; or, the neural network may include multiple network modules (blocks), and each network module includes a multi-layer convolutional layer.
  • the corresponding network module may be determined. Sparse matrix, that is, to determine a corresponding sparse matrix for the multi-layer convolutional layers included in the network module.
  • the structure of the neural network for detecting the target object can be set as required, and this is only an exemplary description.
  • a corresponding sparse matrix may be determined for each layer of convolution modules in the neural network based on the second point cloud data.
  • a corresponding sparse matrix can be determined for each layer of convolution module of the neural network, so that each layer of convolution module can process the input feature map based on the sparse matrix.
  • determining the sparse matrix corresponding to each layer of convolution modules in the neural network may include:
  • S301 Generate an initial sparse matrix based on the second point cloud data.
  • an initial sparse matrix is generated, including:
  • A1 Determine a target area corresponding to the second point cloud data, and divide the target area into a plurality of grid areas according to a preset number of grids.
  • A2 Determine the matrix element value corresponding to each grid area based on the grid area where the point corresponding to the second point cloud data is located.
  • A3 Based on the matrix element value corresponding to each grid area, an initial sparse matrix corresponding to the second point cloud data is generated.
  • the matrix element value of each grid area can be determined. If there is a point corresponding to the second point cloud data, the value of the matrix element of the grid area is 1, indicating that there is a target object at the position of the grid area, and then based on the value of the matrix element corresponding to each grid area, an initial sparse matrix is generated. , which provides data support for the subsequent determination of the 3D detection data of the target object.
  • the target area corresponding to the second point cloud data may be: based on the position when the lidar device acquires the second point cloud data (for example, taking this position as the starting position) and the farthest distance that the lidar device can detect. (eg, with this furthest distance as the length), the resulting detection area is determined.
  • the target area may be determined according to the actual situation in combination with the second point cloud data.
  • the preset number of grids may be N ⁇ M, and the target area may be divided into N ⁇ M grid areas, where N and M are positive integers.
  • the values of N and M can be set according to actual needs.
  • the second point cloud data includes position information of multiple points, and the grid area where each point is located can be determined based on the position information of each point, and further, for each grid area, in the grid area When there is a corresponding point, the value of the matrix element corresponding to the grid area can be 1; when there is no corresponding point in the grid area, the value of the matrix element corresponding to the grid area can be 0, so determine The matrix element value corresponding to each grid area is displayed.
  • an initial sparse matrix corresponding to the second point cloud data may be generated based on the matrix element value corresponding to each grid area, wherein the number of rows of the initial sparse matrix, The number of columns corresponds to the number of grids. For example, if the number of grids is N ⁇ M, the number of rows of the initial sparse matrix is N, and the number of columns is M, that is, the initial sparse matrix is an N ⁇ M matrix.
  • the figure includes a laser radar device 41. Taking the laser radar device as the center, a target area 42 is obtained, and the target area is divided into a plurality of grid areas according to the preset number of grids to obtain The divided grid regions 421 . Then determine the grid area where the multiple points corresponding to the second point cloud data are located, and set the matrix element value of the grid area where the points corresponding to the second point cloud data exist (that is, the grid area with black shadows in the figure). If the value is 1, the matrix element value of the grid area where the point corresponding to the second point cloud data does not exist is set to 0, and the matrix element value of each grid area is obtained. Finally, based on the matrix element value corresponding to each grid area, an initial sparse matrix 43 corresponding to the second point cloud data is generated.
  • the method for determining the sparse matrix corresponding to each layer of convolution modules in the neural network based on the second point cloud data provided by the embodiment of the present disclosure further includes: S302: Based on the initial sparse matrix, determine the sparse matrix input to the neural network. A sparse matrix that matches the target size of the feature maps of each layer of convolutional modules.
  • an initial sparse matrix can be generated based on the second point cloud data, and then based on the initial sparse matrix, a corresponding sparse matrix is determined for each layer of convolution modules of the neural network, and the corresponding sparse matrix of each layer of convolution modules is determined.
  • the matrix matches the target size of the feature map input to the convolution module of this layer, so that each layer of the convolution module can process the input feature map based on the sparse matrix.
  • a sparse matrix matching the target size of the feature map input to the convolution module of each layer of the neural network can be determined.
  • a sparse matrix matching the target size of the feature map input to each layer of the convolution module of the neural network can be determined in the following manner:
  • Mode 1 Determine the output sparse matrix corresponding to each layer of convolution module in the neural network based on the initial sparse matrix, and use the output sparse matrix as the sparse matrix corresponding to the convolution module of this layer.
  • Method 2 Determine the input sparse matrix corresponding to each layer of convolution module in the neural network based on the initial sparse matrix, and use the input sparse matrix as the sparse matrix corresponding to the convolution module of this layer.
  • Method 3 Based on the initial sparse matrix, determine the input sparse matrix and output sparse matrix corresponding to each layer of convolution module in the neural network, fuse the input sparse matrix and the output sparse matrix to obtain a fused sparse matrix, and use the fused sparse matrix as the The sparse matrix corresponding to the layer convolution module.
  • the sparse matrix can be obtained from the output sparse matrix, or can be obtained from the input sparse matrix, or can also be obtained by fusing the input sparse matrix and the output sparse matrix.
  • the sparse matrix can be an input sparse matrix, an output sparse matrix, or a sparse matrix generated based on the input sparse matrix and the output sparse matrix.
  • this mode is to obtain a sparse matrix by outputting a sparse matrix.
  • an output sparse matrix corresponding to each layer of convolution module in the neural network may be determined based on the initial sparse data, and the output sparse matrix is a sparse matrix.
  • the output sparse matrix can be used to represent whether there are corresponding three-dimensional detection data of the target object at different positions of the corresponding target scene in the output result of the convolution module of each layer of the neural network.
  • the matrix at the position corresponding to the position A in the sparse matrix The value may be 1; if the position A does not have the corresponding three-dimensional detection data of the target object, in the sparse matrix, the matrix value at the position corresponding to the position A may be 0.
  • this method is to obtain a sparse matrix from an input sparse matrix.
  • the input sparse matrix corresponding to each layer of convolution module in the neural network may be determined based on the initial sparse data, and the input sparse matrix is a sparse matrix.
  • the input sparse matrix may be 3D detection data representing whether there are corresponding target objects at different positions of the corresponding target scene in the input data of the convolution module of each layer of the neural network.
  • the output sparse matrix corresponding to each layer of convolution module can be determined by mode 1, and the input sparse matrix corresponding to each layer of convolution module can be determined by mode 2, and the input sparse matrix corresponding to each layer of convolution module can be determined. It is fused with the output sparse matrix to obtain a fused sparse matrix, and the fused sparse matrix is used as the sparse matrix corresponding to the convolution module of this layer.
  • the intersection of the input sparse matrix and the output sparse matrix can be taken to obtain a fused sparse matrix; the union of the input sparse matrix and the output sparse matrix can also be taken to obtain the fused sparse matrix.
  • the input sparse matrix is:
  • the input sparse matrix corresponding to each layer of convolution module in the neural network is determined, which may include:
  • i is a positive integer greater than 1 and less than n+1, where n is the total number of layers of the convolution module of the neural network.
  • the initial sparse matrix can be used as the input sparse matrix corresponding to the first-layer convolution module of the neural network.
  • the input sparse matrix corresponding to the second layer convolution module can be obtained from the input sparse matrix corresponding to the first layer convolution module, and the number of rows and columns of the input sparse matrix corresponding to the second layer convolution module is the same as the number of rows and columns input to the second layer
  • the target size of the feature maps of the convolution module is consistent.
  • an image expansion processing operation or an image erosion processing operation can be used to process the input sparse matrix corresponding to the first-layer convolution module to obtain a processed sparse matrix, and the number of rows and columns of the processed sparse matrix is adjusted to After matching the target size of the feature map input to the second-layer convolution module, the input sparse matrix of the second-layer convolution module is obtained.
  • the input sparse matrix corresponding to the first layer convolution module, the input sparse matrix corresponding to the second layer convolution module, ..., the input sparse matrix corresponding to the nth layer convolution module (that is, the last layer of the neural network) can be obtained.
  • input sparse matrix corresponding to the convolution module input sparse matrix corresponding to the convolution module).
  • a dilation processing range may be predetermined, and image dilation processing is performed on the input sparse matrix based on the dilation processing range to obtain a processed sparse matrix, wherein the dilation processing range may be determined based on the size threshold of the target object, or may be determined according to the size threshold of the target object. actually needs to be determined. For example, if the input sparse matrix is:
  • the dilated sparse matrix can be:
  • the erosion process of the input sparse matrix is the inverse process of the expansion process.
  • the erosion process range may be predetermined, and image erosion processing is performed on the input sparse matrix based on the erosion process range to obtain the processed sparse matrix.
  • the corrosion processing range may be determined based on the size threshold of the target object, or may be determined according to actual needs. For example, if the input sparse matrix is:
  • the sparse matrix after erosion processing can be:
  • the number of rows and columns of the processed sparse matrix can be adjusted to a matrix matching the target size of the feature map input to the second-layer convolution module by means of up-sampling or down-sampling, to obtain the first The input sparse matrix of the two-layer convolution module, wherein, there are various processes for adjusting the number of rows and columns of the processed sparse matrix, which are only illustrative here.
  • the sparse degree of the sparse matrix can also be adjusted.
  • the sparse degree of the sparse matrix can be adjusted by adjusting the number of grids; or the sparse degree of the sparse matrix can also be adjusted through the erosion process.
  • the sparse degree of the sparse matrix is: the ratio of the number of matrix elements with a matrix element value of 1 in the sparse matrix to the total number of all matrix elements included in the sparse matrix.
  • the initial sparse matrix can be used as the input sparse matrix corresponding to the first-layer convolution module, and the input sparse matrix of each layer of convolution modules can be determined in turn, and then the sparse matrix can be determined based on the input sparse matrix, which is used for subsequent steps.
  • the 3D detection data of the target object is determined to provide data support.
  • the output sparse matrix corresponding to each layer of convolution module in the neural network which may include:
  • C3 based on the output sparse matrix corresponding to the j+1th layer convolution module, generate an output sparse matrix corresponding to the jth layer convolution module and matching the target size of the feature map input to the jth layer convolution module, wherein, j is a positive integer greater than or equal to 1 and less than n, where n is the total number of layers of the convolution module of the neural network.
  • the processed sparse matrix is the output sparse matrix corresponding to the neural network.
  • the output sparse matrix determine the output sparse matrix of the nth layer convolution module of the neural network (that is, the last layer of the convolution module of the neural network), and so on to obtain the output sparse matrix of the n-1th layer convolution module, ... , the output sparse matrix of the second-layer convolution module, and the output sparse matrix of the first-layer convolution module.
  • the image expansion processing operation or the image erosion processing operation can be used to process the output sparse matrix corresponding to the convolution module of the previous layer to obtain the processed sparse matrix, and the number of rows and columns of the processed sparse matrix can be obtained. After adjusting to match the target size of the feature map input to the convolution module of the current layer, the output sparse matrix of the convolution module of the current layer is obtained.
  • the process of determining the output sparse matrix of the convolution module of each layer reference may be made to the above-mentioned process of determining the input sparse matrix, which will not be described in detail here.
  • the target sparse matrix of each convolution module of the neural network is obtained by the fusion of the input sparse matrix and the output sparse matrix
  • the output sparse matrix and the input sparse matrix of each convolution module of each layer can be obtained by using the above method, respectively.
  • the obtained output sparse matrix and the input sparse matrix are fused to obtain the sparse matrix of each convolution module.
  • the output sparse matrix can be determined based on the initial sparse matrix, and the output sparse matrix of the n-th layer convolution module, .
  • the output sparse matrix of the layer determines the sparse matrix, which provides data support for the subsequent determination of the 3D detection data of the target object based on the sparse matrix of the convolution module of each layer.
  • three-dimensional detection data of the target object included in the target scene may be determined based on the at least one sparse matrix, the second point cloud data, and the neural network for detecting the target object.
  • the three-dimensional detection data includes the coordinates of the center point of the detection frame of the target object, the three-dimensional size of the detection frame, the orientation angle of the detection frame, the category of the target object, the timestamp, and the confidence of the detection frame.
  • the position of the 3D detection frame of the target object cannot exceed the position of the target area, that is, if the coordinates of the center point of the 3D detection frame are (X, Y, Z) and the dimensions are length L, width W, and height H, the following conditions are satisfied Condition: 0 ⁇ X-2/L, X+2/L ⁇ N max , 0 ⁇ YW/2, Y+W/2 ⁇ M max , where N max and M max are the length and width thresholds of the target area .
  • the three-dimensional detection data of the target object included in the target scene is determined, including:
  • Step 1 Based on the second point cloud data, generate a target point cloud feature map corresponding to the second point cloud data.
  • Step 2 Based on the target point cloud feature map and at least one sparse matrix, use a neural network for detecting the target object to determine the three-dimensional detection data of the target object included in the target scene, wherein the neural network includes a multi-layer convolution module.
  • the second point cloud data can be input into the neural network, the second point cloud data can be preprocessed, the target point cloud feature map corresponding to the second point cloud data can be generated, and then the target point cloud feature map, At least one sparse matrix, and a neural network, determine three-dimensional detection data of target objects included in the target scene.
  • step 1 based on the second point cloud data, a target point cloud feature map corresponding to the second point cloud data is generated, which may include:
  • the feature information corresponding to the grid area is determined based on the coordinate information of the point indicated by the second point cloud data located in the grid area; wherein, the grid area is a grid area according to a preset The number is generated by dividing the target area corresponding to the second point cloud data.
  • a target point cloud feature map corresponding to the second point cloud data is generated.
  • the grid area For each grid area, if there is a point corresponding to the second point cloud data in the grid area, the grid area is composed of the coordinate information of each point indicated by the second point cloud data in the grid area Corresponding feature information; if there is no point corresponding to the second point cloud data in the grid area, the feature information of the grid area may be 0.
  • a target point cloud feature map corresponding to the second point cloud data is generated.
  • the size of the target point cloud feature map can be N ⁇ M ⁇ C
  • the size of the target point cloud feature map N ⁇ M is consistent with the size of the sparse matrix of the first-layer convolution module
  • the C of the target point cloud feature map can be is the maximum number of points included in each grid area. For example, if the grid area A has the largest number of points in each grid area, for example, the grid area includes 50 points, the value of C is 50, that is, the target point cloud feature map includes 50 feature maps with a size of N ⁇ M, and each feature map includes coordinate information of at least one point.
  • a target point cloud feature map corresponding to the second point cloud data is generated based on the feature information corresponding to each grid area, and the target point cloud feature map includes the position information of each point, and then based on the target point cloud feature
  • the graph and at least one sparse matrix can more accurately determine the three-dimensional detection data of the target object included in the target scene.
  • step 2 the three-dimensional detection data of the target object included in the target scene can be determined based on the target point cloud feature map, at least one sparse matrix, and the neural network.
  • the three-dimensional detection data of the target object included in the target scene can be determined in the following two ways:
  • Method 1 Based on the target point cloud feature map and at least one sparse matrix, determine the three-dimensional detection data of the target object included in the target scene, including:
  • ⁇ Based on the sparse matrix corresponding to the k-th convolution module in the neural network, determine the feature information to be convolved in the target point cloud feature map input to the k-th convolution module, and use the k-th convolution module of the neural network. , perform convolution processing on the feature information to be convolved in the target point cloud feature map of the kth layer convolution module, and generate a feature map input to the k+1th layer convolution module, where k is a positive value greater than 1 and less than n Integer, n is the number of layers of the convolution module of the neural network.
  • the sparse matrix of the first-layer convolution module can be used to determine the feature information to be convolved in the target point cloud feature map input to the first-layer convolution module.
  • the target position with the matrix value of 1 in the sparse matrix may be determined, and the feature information of the position corresponding to the target position in the target point cloud feature map is determined as the feature information to be convolved.
  • the convolution module of the first layer is used to perform convolution processing on the feature information to be convolved in the feature map of the target point cloud, and the feature map input to the convolution module of the second layer is generated. Then use the sparse matrix of the second-layer convolution module to determine the information to be convolved in the feature map input to the second-layer convolution module, and use the second-layer convolution module to analyze the feature map of the second-layer convolution module.
  • the feature information to be convolved in the convolution process is processed to generate a feature map input to the third-layer convolution module, and so on to obtain the features input to the n-th layer convolution module (the last layer of the convolution module in the neural network).
  • the feature information to be convoluted can be determined based on the sparse matrix of each layer of convolution module and the input feature map, and the feature information to be convolutional is subjected to convolution processing.
  • Other feature information is not processed by convolution, which reduces the calculation amount of convolution processing of each layer of convolution module, improves the operation efficiency of each layer of convolution module, which can reduce the calculation volume of neural network and improve the target object's performance. detection efficiency.
  • Method 2 Based on the target point cloud feature map and at least one sparse matrix, determine the three-dimensional detection data of the target object included in the target scene, including:
  • the convolution vector corresponding to the convolution module of each layer may also be determined based on the target input matrix corresponding to the convolution module of each layer and the feature map input to the convolution module of the layer. For example, for the convolution module of the first layer, the target position with the matrix value of 1 in the sparse matrix of the convolution module of the first layer can be determined, and the feature information of the position corresponding to the target position in the feature map of the target point cloud can be determined. The feature information corresponding to the target position constitutes the convolution vector corresponding to the first-layer convolution module.
  • the img2col and col2img technologies can be used to perform matrix multiplication operations on the corresponding convolution vectors through the first-layer convolution module to obtain a feature map input to the second convolution module.
  • the feature map input to the convolution module of the last layer can be obtained.
  • the convolution vector corresponding to the convolution module of the last layer is determined.
  • the convolution vector corresponding to the one-layer convolution module is processed to determine the three-dimensional detection data of the target object included in the target scene.
  • a convolution vector corresponding to each layer of convolution modules can be generated based on the sparse matrix of each layer of convolution modules and the input feature map, where the convolution vector includes the feature information to be processed in the feature map.
  • the processed feature information is: the feature information in the feature map that matches the position of the three-dimensional detection data of the target object indicated in the sparse matrix, the generated convolution vector is processed, and the features to be processed are removed from the feature map.
  • Other feature information other than the information is not processed, which reduces the calculation amount of convolution processing of each layer of convolution module, improves the operation efficiency of each layer of convolution module, which can reduce the calculation amount of the neural network and improve the target. Object detection efficiency.
  • an embodiment of the present disclosure further provides an intelligent driving control method, including:
  • S501 Use the target radar set on the traveling device to collect point cloud data
  • S502 Perform detection processing on the point cloud data based on the method for processing point cloud data according to any embodiment of the present disclosure, to obtain a detection result;
  • the method for processing point cloud data provided by the embodiments of the present disclosure is easier to deploy in terminal equipment and has higher detection accuracy because the adopted neural network has lower complexity and requires less computing resources.
  • the driving device is, for example, but not limited to, any of the following: an autonomous vehicle, a vehicle equipped with an advanced driving assistance system (Advanced Driving Assistance System, ADAS), a robot, and the like.
  • ADAS Advanced Driving Assistance System
  • Controlling the traveling device includes, for example, controlling the traveling device to accelerate, decelerate, turn, and brake, or play voice prompt information to prompt the driver to control the traveling device to accelerate, decelerate, turn, and brake.
  • the writing order of each step does not mean a strict execution order but constitutes any limitation on the implementation process, and the specific execution order of each step should be based on its function and possible Internal logic is determined.
  • the embodiment of the present disclosure also provides a point cloud data processing device corresponding to the method for processing point cloud data.
  • the processing methods are similar, so the implementation of the device can refer to the implementation of the method, and the repetition will not be repeated.
  • an embodiment of the present disclosure provides an apparatus for processing point cloud data, including an acquisition module 61 , a conversion module 62 and a detection module 63 .
  • the acquisition module 61 is configured to acquire the first point cloud data obtained by the target radar collecting the target scene in the first installation posture.
  • the conversion module 62 is configured to convert the first point cloud data into second point cloud data in a preset standard attitude based on a predetermined first transformation matrix.
  • the detection module 63 performs a detection task on the second point cloud data to obtain a target detection result.
  • the first installation attitude satisfies at least one of the following: a first installation angle of the target radar in the first installation attitude and a standard angle of the target radar in a preset standard attitude The angle difference between them is less than a preset angle difference threshold; the distance between the detection center of the target radar in the first installation attitude and the detection center of the target radar in the preset standard attitude is less than the preset distance threshold.
  • the apparatus for processing point cloud data further includes: a first determination module 64, configured to determine at least one of the angle difference threshold and the distance threshold in the following manner: based on a one-to-many Determine at least one of the angle difference threshold and the distance threshold according to the detection results obtained from the detection of the verification sample sets respectively, wherein each verification sample set includes multiple sets of verification data; the data used for collecting different verification sample sets Radars have different installation attitudes.
  • a first determination module 64 configured to determine at least one of the angle difference threshold and the distance threshold in the following manner: based on a one-to-many Determine at least one of the angle difference threshold and the distance threshold according to the detection results obtained from the detection of the verification sample sets respectively, wherein each verification sample set includes multiple sets of verification data; the data used for collecting different verification sample sets Radars have different installation attitudes.
  • the device for processing point cloud data further includes: a second determination module 65, configured to determine the first transformation matrix in the following manner: based on the target radar in the first installation attitude The actual detection data and the predetermined standard detection data of the target radar in the preset standard attitude are used to determine the first transformation matrix.
  • the detection module 63 when performing a detection task on the second point cloud data to obtain a target detection result, is used to: generate the second point cloud data based on the second point cloud data.
  • a sparse matrix of at least one target corresponding to the point cloud data; the sparse matrix is used to represent whether there are target objects at different positions of the target scene; based on the at least one sparse matrix and the second point cloud data, determine The three-dimensional detection data of the target object included in the target scene; the three-dimensional detection data is used as the target detection result.
  • the detection module 63 when generating at least one sparse matrix corresponding to the second point cloud data based on the second point cloud data, is configured to: based on the second point cloud data data, and determine the sparse matrix corresponding to each layer of convolution modules in the neural network for detecting the target object.
  • the detection module 63 when determining, based on the second point cloud data, the sparse matrix corresponding to each layer of convolution modules in the neural network for detecting the target object, uses In: generating an initial sparse matrix based on the second point cloud data; and determining, based on the initial sparse matrix, a sparse matrix matching the target size of the feature map input to each layer of the convolution module of the neural network.
  • the detection module 63 when generating the initial sparse matrix based on the second point cloud data, is used to: determine the target area corresponding to the second point cloud data, and according to the preset The target area is divided into a plurality of grid areas; based on the grid area where the point corresponding to the second point cloud data is located, the matrix element value corresponding to each grid area is determined; based on each grid area The matrix element values corresponding to the grid regions are generated, and the initial sparse matrix corresponding to the second point cloud data is generated.
  • the detection module 63 when determining, based on the initial sparse matrix, a sparse matrix that matches the target size of the feature map input to the convolution module of each layer of the neural network, uses In any of the following: based on the initial sparse matrix, determine the output sparse matrix corresponding to each layer of convolution module in the neural network, and use the output sparse matrix as the sparse matrix corresponding to the convolution module of this layer; the initial sparse matrix, determine the input sparse matrix corresponding to each layer of convolution module in the neural network, and use the input sparse matrix as the sparse matrix corresponding to the convolution module of this layer; based on the initial sparse matrix, determine the The input sparse matrix and the output sparse matrix corresponding to each layer of convolution module in the neural network are fused, the input sparse matrix and the output sparse matrix are fused to obtain a fused sparse matrix, and the fused sparse matrix is used as the convolution module
  • the detection module 63 determines, based on the initial sparse matrix, that the input sparse matrix corresponding to each layer of the convolution module in the neural network is, for: converting the initial sparse matrix As the input sparse matrix corresponding to the convolution module of the first layer of the neural network; The input sparse matrix matching the target size of the feature map of the layer convolution module; wherein, i is a positive integer greater than 1 and less than n+1, and n is the total number of layers of the convolution module of the neural network.
  • the detection module 63 when determining the output sparse matrix corresponding to each layer of convolution module in the neural network based on the initial sparse matrix, is used to: The size threshold and the initial sparse matrix determine the output sparse matrix corresponding to the neural network; based on the output sparse matrix, generate the features corresponding to the nth layer convolution module and input to the nth layer convolution module The output sparse matrix matching the target size of the graph; based on the output sparse matrix corresponding to the j+1th layer convolution module, generate the feature map corresponding to the jth layer convolution module and input to the jth layer convolution module.
  • the output sparse matrix matching the target size where j is a positive integer greater than or equal to 1 and less than n, where n is the total number of layers of the convolution module of the neural network.
  • the detection module 63 when determining the three-dimensional detection data of the target object included in the target scene based on the at least one sparse matrix and the second point cloud data, is used to: : based on the second point cloud data, generate a point cloud feature map corresponding to the second point cloud data; based on the point cloud feature map and the at least one sparse matrix, use a neural network for detecting the target object to determine the three-dimensional detection data of the target object included in the target scene, wherein the neural network includes a multi-layer convolution module.
  • the detection module 63 when generating the point cloud feature map corresponding to the second point cloud data based on the second point cloud data, is used for: for each grid area, Based on the coordinate information of the point indicated by the second point cloud data located in the grid area, the characteristic information corresponding to the grid area is determined; The target area corresponding to the second point cloud data is divided and generated; based on the feature information corresponding to each grid area, a point cloud feature map corresponding to the second point cloud data is generated.
  • the detection module 63 uses a neural network for detecting target objects based on the point cloud feature map and the at least one sparse matrix to determine the three-dimensional detection of the target objects included in the target scene.
  • the data is used, it is used to: determine the feature information to be convolved in the point cloud feature map based on the sparse matrix corresponding to the first-layer convolution module in the neural network, and use the first-layer convolution module to analyze the
  • the feature information to be convolved in the point cloud feature map is subjected to convolution processing to generate a feature map input to the second-layer convolution module; based on the sparse matrix corresponding to the k-th layer convolution module in the neural network, determine Input to the feature information to be convoluted in the feature map of the k-th layer convolution module, use the k-th layer convolution module of the neural network to be convoluted in the feature map of the k-th layer convolution module.
  • n is the total number of layers of the convolution module of the neural network; based on The sparse matrix corresponding to the nth layer convolution module in the neural network determines the feature information to be convolved in the feature map input to the nth layer convolution module, and uses the nth layer convolution module of the neural network. , performing convolution processing on the feature information to be convolved in the feature map of the n-th layer convolution module to obtain three-dimensional detection data of the target object included in the target scene.
  • the detection module 63 uses a neural network for detecting target objects to determine the three-dimensional dimensions of the target objects included in the target scene.
  • detecting data it is used for: for each other convolution module in the neural network except the convolution module of the last layer, based on the sparse matrix corresponding to the convolution module of this layer and the input to the convolution module of this layer.
  • an embodiment of the present disclosure further provides an intelligent driving control device, including:
  • an acquisition module 71 configured to acquire point cloud data collected by a target radar set on the traveling device
  • a processing module 72 configured to perform detection processing on the point cloud data based on the processing method for point cloud data described in any embodiment of the present disclosure, to obtain a detection result;
  • the control module 73 is configured to control the traveling device based on the detection result.
  • an embodiment of the present disclosure further provides a computer device, including a processor 11 and a memory 12; the memory 12 stores machine-readable instructions executable by the processor 11, and when the computer device runs , the machine-readable instructions are executed by the processor to implement the following steps:
  • a detection task is performed on the second point cloud data to obtain a target detection result.
  • machine-readable instructions are executed by the processor to implement the following steps:
  • the traveling device is controlled.
  • Embodiments of the present disclosure further provide a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is run by a processor, the method for processing point cloud data described in the foregoing method embodiments or Intelligent driving control method.
  • the storage medium may be a volatile or non-volatile computer-readable storage medium.
  • the computer program product of the point cloud data processing method or the intelligent driving control method provided by the embodiments of the present disclosure includes a computer-readable storage medium storing program codes, and the instructions included in the program codes can be used to execute the above method embodiments.
  • the steps of the point cloud data processing method or the intelligent driving control method reference may be made to the above method embodiments, which will not be repeated here.
  • Embodiments of the present disclosure also provide a computer program, which implements any one of the methods in the foregoing embodiments when the computer program is executed by a processor.
  • the computer program product can be specifically implemented by hardware, software or a combination thereof.
  • the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (Software Development Kit, SDK), etc. Wait.
  • the units described as separate components may or may not be physically separated, and components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.
  • each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit.
  • the functions, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a processor-executable non-volatile computer-readable storage medium.
  • the computer software products are stored in a storage medium, including Several instructions are used to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present disclosure.
  • the aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk and other media that can store program codes .

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Aviation & Aerospace Engineering (AREA)
  • Electromagnetism (AREA)
  • Automation & Control Theory (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Traffic Control Systems (AREA)
  • Radar Systems Or Details Thereof (AREA)

Abstract

一种点云数据的处理方法、智能行驶控制方法及装置，其中，该方法包括：获取目标雷达在第一安装姿态下采集目标场景得到的第一点云数据（S101）；基于预先确定的第一变换矩阵，将第一点云数据转换为预设标准姿态下的第二点云数据（S102）；对第二点云数据执行检测任务，得到目标检测结果（S103）。

Description

点云数据的处理方法、智能行驶控制方法及装置
相关申请的交叉引用
本专利申请要求于2020年7月22日提交的、申请号为202010713992.8、发明名称为“点云数据的处理方法、智能行驶控制方法及装置”的中国专利申请的优先权，该申请以引用的方式并入本文中。
技术领域
本公开涉及数据处理技术领域，具体而言，涉及一种点云数据的处理方法、智能行驶控制方法及装置。
背景技术
激光雷达以其精确的测距能力，被广泛用于自动驾驶、无人机勘探、地图测绘等领域。基于激光雷达提供的点云数据，产生了如目标检测、建图、定位、点云分割、场景流等各类应用。对于一些神经网络或者深度学习算法而言，为了解决由于点云数据的分布不同所带来的模型精度下降的问题，通常会采用分布不同的大量样本数据来训练模型；但由于要求算法对分布不同的点云数据都要有良好的处理能力，导致算法的设计难度大幅增加。
发明内容
本公开实施例至少提供一种点云数据的处理方法、智能行驶控制方法、装置、计算机设备及存储介质。
第一方面，本公开实施例提供了一种点云数据的处理方法，包括：获取目标雷达在第一安装姿态下采集目标场景得到的第一点云数据；基于预先确定的第一变换矩阵，将所述第一点云数据转换为预设标准姿态下的第二点云数据；对所述第二点云数据执行检测任务，得到目标检测结果。
这样，利用预先确定的第一变换矩阵，将目标雷达在第一安装姿态下采集的目标场景的第一点云数据，转换为预设标准姿态下的第二点云数据，并对第二点云数据执行检测任务，得到目标检测结果，从而能够将不同安装姿态下的点云数据都转换至标准安装姿态下执行检测任务，从而执行检测任务的算法只需要对标准安装姿态下的点云数据具有良好处理能力即可，降低算法的设计难度。
一种可能的实施方式中，所述第一安装姿态满足以下中的至少一个：所述目标雷达在第一安装姿态时的第一安装角度与所述目标雷达在所述预设标准姿态时的标准角度之间的角度差小于预设的角度差阈值；所述目标雷达在所述第一安装姿态下的检测中心与所述目标雷达在所述预设标准姿态下的检测中心之间的距离小于预设的距离阈值。
这样，将目标雷达的第一安装姿态限定在预设标准姿态附近，从而使得在第一安装姿态下获得的第一点云数据的特征分布和在预设标准姿态下获得的点云数据的特征分布接近，进而对目标雷达的安装姿态不会过度的限制，且保证了目标检测结果的精度。
一种可能的实施方式中，采用下述方式确定所述角度差阈值和所述距离阈值中的至少一个：基于对多个验证样本集分别进行检测所得到的检测结果，确定所述角度差阈值和所述距离阈值中的至少一个，其中，每个验证样本集中包括多组验证数据；用于采集不同验证样本集的雷达具有不同的安装姿态。
这样，通过验证的方式，确定角度差阈值和距离阈值中的至少一个，从而能够让得到的角度差阈值和距离阈值中的至少一个具有更高的精度，保证了目标检测结果的精度。
一种可能的实施方式中，用下述方式确定所述第一变换矩阵：基于所述目标雷达在所述第一安装姿态下的实际检测数据、以及预先确定的所述目标雷达在所述预设标准姿态下的标准检测数据，确定所述第一变换矩阵。
一种可能的实施方式中，所述对所述第二点云数据执行检测任务，得到目标检测结果，包括：基于所述第二点云数据，生成所述第二点云数据对应的至少一个稀疏矩阵；所述稀疏矩阵用于表征所述目标场景的不同位置处是否存在目标对象；基于所述至少一个稀疏矩阵、和所述第二点云数据，确定所述目标场景中包括的目标对象的三维检测数据；将所述三维检测数据作为所述目标检测结果。
这样，可以为获取到的第二点云数据生成对应的至少一个稀疏矩阵，该稀疏矩阵用于表征目标场景的不同位置处是否具有目标对象；这样，在基于稀疏矩阵和第二点云数据，确定目标对象的三维检测数据时，可以基于稀疏矩阵，确定存在对应的目标对象的目标位置，从而可以将与该目标位置对应的特征进行处理，将不同位置中除目标位置之外的其他位置对应的特征不进行处理，这样就减少了得到目标对象的三维检测数据的计算量，提高了检测效率。
一种可能的实施方式中，基于所述第二点云数据，生成所述第二点云数据对应的至少一个稀疏矩阵，包括：基于所述第二点云数据，确定用于检测所述目标对象的神经网络中每一层卷积模块所对应的稀疏矩阵。
这样，可以基于第二点云数据，为神经网络的每一层卷积模块确定对应的稀疏矩阵，使得每一层卷积模块可以基于稀疏矩阵对输入的特征图(feature map)进行处理。
一种可能的实施方式中，基于所述第二点云数据，确定所述神经网络中每一层卷积模块所对应的稀疏矩阵，包括：基于所述第二点云数据，生成初始稀疏矩阵；基于所述初始稀疏矩阵，确定与输入至所述神经网络的每一层卷积模块的特征图的目标尺寸匹配的稀疏矩阵。
这样，可以基于第二点云数据，生成初始稀疏矩阵，再基于初始稀疏矩阵，为神经网络的每一层卷积模块确定对应的稀疏矩阵，且每一层卷积模块对应的稀疏矩阵与输入至该层卷积模块的特征图的目标尺寸相匹配，使得每一层卷积模块可以基于稀疏矩阵对输入的特征图进行处理。
一种可能的实施方式中，基于所述第二点云数据，生成初始稀疏矩阵，包括：确定所述第二点云数据对应的目标区域，并按照预设的栅格数量，将所述目标区域划分为多个栅格区域；基于所述第二点云数据对应的点所处的栅格区域，确定每个栅格区域对应的矩阵元素值；基于每个栅格区域对应的矩阵元素值，生成所述第二点云数据对应的初始稀疏矩阵。
这样，可以基于第二点云数据，判断每个栅格区域中是否存在第二点云数据对应的点，基于判断结果，确定每个栅格区域的矩阵元素值，比如，若栅格区域中存在第二点云数据对应的点，则该栅格区域的矩阵元素值为1，表征该栅格区域位置处存在目标对象，进而基于各个栅格区域对应的矩阵元素值，生成了初始稀疏矩阵，为后续确定目标对象的三维检测数据提供了数据支持。
一种可能的实施方式中，基于所述初始稀疏矩阵，确定与输入至所述神经网络的每一层卷积模块的特征图的目标尺寸匹配的稀疏矩阵，包括以下任一：基于所述初始稀疏矩阵，确定所述神经网络中，每一层卷积模块对应的输出稀疏矩阵，将该输出稀疏矩阵作为该层卷积模块对应的所述稀疏矩阵；基于所述初始稀疏矩阵，确定所述神经网络中，每一层卷积模块对应的输入稀疏矩阵，将该输入稀疏矩阵作为该层卷积模块对应的所述稀疏矩阵；基于所述初始稀疏矩阵，确定所述神经网络中，每一层卷积模块对应的输入稀疏矩阵和输出稀疏矩阵，将所述输入稀疏矩阵和输出稀疏矩阵进行融合，得到融合稀疏矩阵，将所述融合稀疏矩阵作为该层卷积模块对应的稀疏矩阵。
这样，设置多种方式，生成每一层卷积模块对应的稀疏矩阵，即稀疏矩阵可以为输 入稀疏矩阵，也可以为输出稀疏矩阵，还可以为基于输入稀疏矩阵和输出稀疏矩阵生成的融合稀疏矩阵。
一种可能的实施方式中，基于所述初始稀疏矩阵，确定所述神经网络中，每一层卷积模块对应的输入稀疏矩阵，包括：将所述初始稀疏矩阵作为所述神经网络的第一层卷积模块对应的输入稀疏矩阵；基于第i-1层卷积模块对应的输入稀疏矩阵，确定第i层卷积模块对应的、与输入至所述第i层卷积模块的特征图的目标尺寸匹配的输入稀疏矩阵；其中，i为大于1、且小于n+1的正整数，n为所述神经网络的卷积模块的总层数。
这样，可以将初始稀疏矩阵作为第一层卷积模块对应的输入稀疏矩阵，并依次确定得到每一层卷积模块的输入稀疏矩阵，进而可以基于该输入稀疏矩阵确定稀疏矩阵，为后续基于每一层卷积模块的稀疏矩阵，确定目标对象的三维检测数据提供了数据支持。
一种可能的实施方式中，所述基于所述初始稀疏矩阵，确定所述神经网络中，每一层卷积模块对应的输出稀疏矩阵，包括：基于所述目标对象的尺寸阈值和所述初始稀疏矩阵，确定所述神经网络对应的输出稀疏矩阵；基于所述输出稀疏矩阵，生成第n层卷积模块对应的、与输入至所述第n层卷积模块的特征图的目标尺寸匹配的输出稀疏矩阵；基于第j+1层卷积模块对应的输出稀疏矩阵，生成第j层卷积模块对应的、与输入至所述第j层卷积模块的特征图的目标尺寸匹配的输出稀疏矩阵，其中，j为大于等于1、且小于n的正整数，n为所述神经网络的卷积模块的总层数。
这样，可以基于初始稀疏矩阵，确定输出稀疏矩阵，利用输出稀疏矩阵依次确定第n层卷积模块的输出稀疏矩阵、…、第一层卷积模块的输出稀疏矩阵，进而可以基于每一层的输出稀疏矩阵确定稀疏矩阵，为后续基于每一层卷积模块的稀疏矩阵，确定目标对象的三维检测数据提供了数据支持。
一种可能的实施方式中，基于所述至少一个稀疏矩阵、和所述第二点云数据，确定所述目标场景中包括的目标对象的三维检测数据，包括：基于所述第二点云数据，生成所述第二点云数据对应的目标点云特征图；基于所述目标点云特征图和所述至少一个稀疏矩阵，利用检测目标对象的神经网络，确定所述目标场景中包括的目标对象的三维检测数据，其中，所述神经网络中包括多层卷积模块。
一种可能的实施方式中，基于所述第二点云数据，生成所述第二点云数据对应的目标点云特征图，包括：针对每个栅格区域，基于位于所述栅格区域内的所述第二点云数据所指示的点的坐标信息，确定所述栅格区域对应的特征信息；其中，所述栅格区域为按照预设的栅格数量，将所述第二点云数据对应的目标区域划分生成的；基于每个栅格区域对应的特征信息，生成所述第二点云数据对应的目标点云特征图。
这样，基于每个栅格区域对应的特征信息，生成了第二点云数据对应的目标点云特征图，目标点云特征图中包括每个点的位置信息，进而基于目标点云特征图和至少一个稀疏矩阵，可以较准确的确定目标场景中包括的目标对象的三维检测数据。
一种可能的实施方式中，基于所述目标点云特征图和所述至少一个稀疏矩阵，利用检测目标对象的神经网络，确定所述目标场景中包括的目标对象的三维检测数据，包括：基于所述神经网络中第一层卷积模块对应的稀疏矩阵，确定所述目标点云特征图中的待卷积特征信息，利用所述第一层卷积模块，对所述目标点云特征图中的所述待卷积特征信息进行卷积处理，生成输入至第二层卷积模块的特征图；基于所述神经网络中第k层卷积模块对应的稀疏矩阵，确定输入至所述第k层卷积模块的特征图中的待卷积特征信息，利用所述神经网络的第k层卷积模块，对所述第k层卷积模块的特征图中的待卷积特征信息进行卷积处理，生成输入至第k+1层卷积模块的特征图，其中，k为大于1、小于n的正整数，n为所述神经网络的卷积模块的总层数；基于所述神经网络中第n层卷积模块对应的稀疏矩阵，确定输入至所述第n层卷积模块的特征图中的待卷积特征信息，利用所述神经网络的第n层卷积模块，对所述第n层卷积模块的特征图中的待卷积特征信息进行卷积处理，得到所述目标场景中包括的目标对象的三维检测数据。
这样，可以基于每一层卷积模块的稀疏矩阵和输入的特征图，确定待卷积特征信息，对待卷积特征信息进行卷积处理，对特征图中除待卷积特征信息之外的其他特征信息不进行卷积处理，减少了每一层卷积模块进行卷积处理的计算量，提高了每一层卷积模块的运算效率，进而可以减少神经网络的运算量，提高目标对象的检测效率。
一种可能的实施方式中，基于所述目标点云特征图和所述至少一个稀疏矩阵，利用检测目标对象的神经网络，确定所述目标场景中包括的目标对象的三维检测数据，包括：针对所述神经网络中除最后一层卷积模块之外的其他每一层卷积模块，基于该层卷积模块对应的稀疏矩阵和输入至该层卷积模块的特征图，确定该层卷积模块对应的卷积向量；基于该层卷积模块对应的所述卷积向量，确定输入至下一层卷积模块的特征图；基于最后一层卷积模块对应的稀疏矩阵和输入至最后一层卷积模块的特征图，确定最后一层卷积模块对应的卷积向量；基于最后一层卷积模块对应的所述卷积向量，确定所述目标场景中包括的目标对象的三维检测数据。
这样，可以基于每一层卷积模块的稀疏矩阵和输入的特征图，生成每一层卷积模块对应的卷积向量，该卷积向量中包括特征图中的待处理的特征信息，该待处理的特征信息为：与稀疏矩阵中指示的存在目标对象的三维检测数据的位置匹配的、特征图中的特征信息，对生成的卷积向量进行处理，而对特征图中除待处理的特征信息之外的其他特征信息不进行处理，减少了每一层卷积模块进行卷积处理的计算量，提高了每一层卷积模块的运算效率，进而可以减少神经网络的运算量，提高目标对象的检测效率。
第二方面，本公开实施例提供一种智能行驶控制方法，包括：利用设置在行驶装置上的目标雷达采集点云数据；基于第一方面任一项所述的点云数据的处理方法对所述点云数据进行检测处理，得到检测结果；基于所述检测结果，控制所述行驶装置。
本公开实施例提供的智能行驶控制方法，在获取目标雷达采集的点云数据后，会将目标雷达采集的点云数据转换至标注安装姿态下进行检测处理，得到检测结果，从而执行检测任务的算法只需要对标准安装姿态下的点云数据具有良好处理能力即可，降低算法的设计难度。
第三方面，本公开实施例提供一种点云数据的处理装置，包括：获取模块，用于获取目标雷达在第一安装姿态下采集目标场景得到的第一点云数据；转换模块，用于基于预先确定的第一变换矩阵，将所述第一点云数据转换为预设标准姿态下的第二点云数据；检测模块，用于对所述第二点云数据执行检测任务，得到目标检测结果。
第四方面，本公开实施例还提供一种智能行驶控制装置，包括：获取模块，用于获取设置在行驶装置上的目标雷达采集的点云数据；处理模块，用于基于第一方面任一项所述的点云数据的处理方法对所述点云数据进行检测处理，得到检测结果；控制模块，用于基于所述检测结果，控制所述行驶装置。
第五方面，本公开可选实现方式还提供一种计算机设备，包括处理器和存储器，所述存储器存储有所述处理器可执行的机器可读指令，所述机器可读指令被所述处理器执行时，所述机器可读指令被所述处理器执行时执行上述第一方面，或第一方面中任一种可能的实施方式中的步骤；或执行上述第二方面的实施方式中的步骤。
第六方面，本公开可选实现方式还提供一种计算机可读存储介质，其上存储的计算机程序被运行时执行上述第一方面，或第一方面中任一种可能的实施方式中的步骤；或执行上述第二方面的实施方式中的步骤。
附图说明
图1示出了本公开实施例所提供的一种点云数据的处理方法的流程图；
图2示出了本公开实施例所提供的第二点云数据执行检测任务，得到目标检测结果的具体方法的流程图；
图3示出了本公开实施例所提供的基于第二点云数据，确定神经网络中每一层卷积 模块对应的稀疏矩阵的具体方法的流程图；
图4示出了本公开实施例所提供的一种目标区域和该目标区域对应的初始稀疏矩阵的示意图；
图5示出了本公开实施例所提供的一种智能行驶控制方法的流程图；
图6示出了本公开实施例所提供的一种点云数据的处理装置的示意图；
图7示出了本公开实施例所提供的一种智能行驶控制装置的示意图；
图8示出了本公开实施例所提供的一种计算机设备的示意图。
具体实施方式
为使本公开实施例的目的、技术方案和优点更加清楚，下面将结合本公开实施例中附图，对本公开实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本公开一部分实施例，而不是全部的实施例。通常在此处附图中描述和示出的本公开实施例的组件可以以各种不同的配置来布置和设计。因此，以下对在附图中提供的本公开的实施例的详细描述并非旨在限制要求保护的本公开的范围，而是仅仅表示本公开的选定实施例。基于本公开的实施例，本领域技术人员在没有做出创造性劳动的前提下所获得的所有其他实施例，都属于本公开保护的范围。
经研究发现，在利用神经网络或者深度学习算法执行如基于三维点云的目标检测、建图、点云分割、场景流等各类数据处理任务时，需要预先获得具有一些特定特征的样本点云数据，并利用样本点云数据训练神经网络模型，使得神经网络模型能够学习到点云数据的特定特征，从而能够利用已训练的神经网络模型执行对应的数据处理任务。但对于三维点云数据而言，用于获取三维点云数据的激光雷达具有不同的安装姿态，不同位姿分别对应的三维点云数据之间就会有特征分布的差异。为了让神经网络模型能够适应不同安装姿态的激光雷达，当前一般会采用不同安装姿态的激光雷达分别获取样本数据，并基于多个激光雷达获取的样本数据来训练模型，让神经网络模型能够充分学习到特征分布不同的三维点云数据分别对应的特征。但神经网络模型需要学习的特征越多，对应的算法也就越复杂，造成算法设计难度的大幅增加。
此外，由于算法的复杂度增加，还会导致基于神经网络模型执行数据处理任务时所需要消耗的计算资源的增加，因而导致了硬件成本的增加。
基于上述研究，本公开提供了一种点云数据的处理方法及装置，利用预先确定的第一变换矩阵，将目标雷达在第一安装姿态下采集的目标场景的第一点云数据转换为标准姿态下的第二点云数据，并对第二点云数据执行检测任务，得到目标检测结果，从而能够将不同安装姿态下的点云数据都转换至标准安装姿态下来执行检测任务，从而执行检测任务的算法只需要对标准安装姿态下的点云数据具有良好处理能力即可，降低算法的设计难度。
针对以上方案所存在的缺陷，均是发明人在经过实践并仔细研究后得出的结果，因此，上述问题的发现过程以及下文中本公开针对上述问题所提出的解决方案，都应该是发明人在本公开过程中对本公开做出的贡献。
应注意到：相似的标号和字母在下面的附图中表示类似项，因此，一旦某一项在一个附图中被定义，则在随后的附图中不需要对其进行进一步定义和解释。
为便于对本实施例进行理解，首先对本公开实施例所公开的一种点云数据的处理方法进行详细介绍，本公开实施例所提供的点云数据的处理方法的执行主体一般为数据处理设备，该数据处理设备例如包括：终端设备或服务器或其它处理设备，终端设备可以为用户设备(User Equipment，UE)、移动设备、用户终端、终端、蜂窝电话、无绳电话、个人数字处理(Personal Digital Assistant，PDA)、手持设备、计算设备、车载设备、可穿戴设备等。在一些可能的实现方式中，该点云数据的处理方法可以通过处理器调用存储器中存储的计算机可读指令的方式来实现。
本公开实施例所提供的点云数据的处理方法能够应用于多个领域，例如智能行驶领域、智能机器人、无人机、增强现实(Augmented Reality，AR)、虚拟现实(Virtual Reality，VR)、海洋探测、3D打印等领域。下面以应用领域为智能行驶为例，对本公开实施例提供的点云数据的处理方法加以说明。
参见图1所示，本公开实施例提供一种点云数据的处理方法，所述方法包括步骤S101至S103，其中：
S101：获取目标雷达在第一安装姿态下采集目标场景得到的第一点云数据。
S102：基于预先确定的第一变换矩阵，将所述第一点云数据转换为预设标准姿态下的第二点云数据。
S103：对所述第二点云数据执行检测任务，得到目标检测结果。
本公开实施例利用预先确定的第一变换矩阵，将目标雷达在第一安装姿态下采集的目标场景的第一点云数据转换为标准姿态下的第二点云数据，并对第二点云数据执行检测任务，得到目标检测结果，从而能够将不同安装姿态下的点云数据都转换至标准安装姿态下执行检测任务，使得执行检测任务的算法只需要对标准安装姿态下的点云数据具有良好处理能力即可，降低算法的设计难度。
同时，由于算法的复杂度更低，基于算法确定的神经网络结构也相应简单，进而神经网络的体积也相应更小，更适于部署在嵌入式设备中。
另外，算法简单的神经网络在执行任务处理时所需要消耗的计算资源也相应减少，降低硬件成本。
下面分别对上述S101至S103加以详细说明。
针对上述S101：
其中，以目标雷达为安装在智能行驶设备中的激光雷达为例，目标雷达有线或者无线连接数据处理设备。数据处理设备能够对目标雷达在其第一安装姿态下的第一点云数据进行后续的处理。
示例性的，目标雷达的第一安装姿态是指目标雷达实际安装在智能行驶设备中时的安装姿态。
预设标准姿态是指为目标雷达确定的在智能行驶设备中的基准安装姿态。例如，目标雷达的检测中心位于智能行驶设备中的A位置，且目标雷达的检测轴线与预设的某个轴线L重合时，目标雷达处于预设标准姿态。此时，以A位置为原点建立基准坐标系。基准坐标系的z轴例如为轴线L，A位置所在的空间点通过该轴线L；x轴和y轴所在的平面垂直于z轴，且A位置所在的点位于该平面上。
目标雷达在实际安装时，位于智能行驶设备中的B位置(例如目标雷达的检测中心位于B位置)，则基于目标雷达在实际安装中的第一安装姿态建立雷达坐标系，雷达坐标系的原点为雷达的检测中心，z轴为雷达的检测轴线，x轴和y轴所在的平面垂直于z轴，且B位置所在的点位于x轴和y轴所在的平面上。
由于目标雷达在实际中的第一安装姿态和预设标准姿态存在差异，因此，相对于预设标准姿态而言，目标雷达存在一定角度的偏差和一定距离的位移。
在角度的偏差为0，位移也为0的时候，认为目标雷达在实际安装中的第一安装姿态和预设标准姿态一致。
示例性的，假设以水平方向作为预设的安装基准，以目标雷达位于预设安装位置时的检测中心为原点，以在水平方向上的直线作为z轴，以垂直于z轴的平面为x轴和y轴所在的平面，建立基准坐标系，则目标雷达的第一安装姿态例如可以通过目标雷达的检测轴线与x、y和z轴之间的夹角、以及目标雷达的检测中心相对于原点的位移来表征，例如可以表示为：(ρ,
Figure PCTCN2021102706-appb-000001
θ,x 0,y 0,z 0)。其中，ρ表示检测轴线和基准坐标系的x轴之间的夹角；
Figure PCTCN2021102706-appb-000002
表示检测轴线和基准坐标系的y轴之间的夹角，θ表示检测轴线和基准 坐标系的z轴之间的夹角，x 0,y 0,z 0表示目标雷达在位于第一安装姿态时，检测中心在基准坐标系中的坐标值。
为了保证目标雷达所获取的第一点云数据的特征分布与预设标准安装姿态下获取的第二点云数据的特征分布尽可能接近，本公开另一实施例中，第一安装姿态满足以下中的至少一个：第一安装姿态对应的安装角度与预设标准姿态对应的标准角度之间的第一角度差小于预设的角度差阈值；第一安装姿态对应的检测中心与所述预设标准姿态对应的检测中心之间的距离小于预设的距离阈值。
在一种可能的情况下，目标雷达在实际安装中的第一安装姿态下，检测中心和基准坐标系的圆心完全重合，则此时，第一安装姿态对应的检测中心与所述预设标准姿态对应的检测中心之间的距离等于0。
若目标雷达在实际安装中的第一安装姿态下，第一安装姿态对应的安装角度与预设标准姿态对应的标准角度之间的第一角度差等于0。此处，角度差阈值例如可以表示为：(Δρ,
Figure PCTCN2021102706-appb-000003
Δθ)，距离阈值例如可以表示为：(Δx,Δy,Δz)。
这样，在利用神经网络对第二点云数据执行检测任务的时候，通过对目标雷达的第一安装姿态进行一定的限定，从而保证了第二点云数据和训练神经网络的训练样本的特征分布尽量接近，进而保证了神经网络所需要学习的特征不会过多，降低神经网络的设计难度和训练得到的神经网络的复杂度，使得到的神经网络的体积更小，更适于部署在嵌入式设备中，神经网络在执行检测任务时所需要的计算资源也就更少，降低硬件成本。
本公开实施例提供一种确定角度差阈值和距离阈值中的至少一个的具体方法，包括：基于对多个验证样本集分别进行检测得到的检测结果，确定所述角度差阈值和所述距离阈值中的至少一个，其中，每个验证样本集中包括多组验证数据；用于采集不同验证样本集的雷达具有不同的安装姿态。
在具体实施中，例如可以利用神经网络对多个验证样本集分别执行检测任务，得到多个验证样本集分别对应的检测结果；针对多个验证样本集中的每个验证样本集，每个验证样本集中包括了多组验证数据；其中同一验证样本集中的多组验证数据来源于同一验证雷达。在获取验证样本集时，可以随机的确定验证雷达的安装姿态，然后在该安装姿态下利用验证雷达获取多组验证数据。然后使用验证雷达在不同的安装姿态下获取的多组验证数据，确定角度差阈值和距离阈值中的至少一个。
此处，获取不同验证数据集的验证雷达可以为同一种雷达，也可以是不同的雷达。
在一种可能的实施方式中，在目标雷达的型号、参数等数据确定的情况下，可以采用同样类型的雷达获取训练样本，相应地，采用相同类型的雷达获取验证样本集。
在使用验证雷达在不同的安装姿态下获取的多组验证数据，确定角度差阈值的时候，例如可以根据神经网络每个验证样本集中的多组验证数据进行检测任务时分别对应的检测结果，确定与每个验证样本集对应的检测损失，然后基于多个验证样本集分别对应的检测损失，确定角度差阈值。
此处，以确定角度差阈值为例，例如可以从多个验证验证样本集中，确定检测损失小于一定损失阈值的多个验证样本集，将确定的多个验证样本集分别对应的第二角度差中的最大值确定为角度差阈值；或者，将确定的多个验证样本集分别对应的第二角度差的平均值作为角度差阈值。
在另一种可能的实施方式中，还可以预先确定多个验证区间，第i个验证区间例如可以表示为：
Figure PCTCN2021102706-appb-000004
和/或
Figure PCTCN2021102706-appb-000005
仍然以确定角度差为例，在每个验证区间下获取多组验证样本集，然后利用预先训 练的神经网络对每个验证区间下的多组验证样本集分别进行执行检测任务，并基于每个验证区间下的多组验证样本集分别对应的检测结果，确定每个验证区间分别对应的检测损失，并基于每个验证区间所对应的验证损失，确定角度差阈值。
确定距离阈值的方式与确定角度差阈值的方式类似，在此不再赘述。
针对上述S102：
第一变换矩阵是将第一点云数据从第一安装姿态下变换至预设标准姿态下的转换矩阵。
示例性地，基于目标雷达的第一安装位置建立雷达坐标系，目标雷达的在第一安装姿态下的第一点云数据中，点云的任一点在基于目标雷达建立的雷达坐标系中的坐标值为：(x,y,z)，该点在基准坐标系中的坐标值为(x′,y′,z′)，则(x,y,z)和(x′,y′,z′)之间满足下述公式(1)：
Figure PCTCN2021102706-appb-000006
其中，r ij为旋转参数，t k为平移参数。
若目标雷达的第一安装位置和预设基准位置完全重合，则满足下述公式(2)：
Figure PCTCN2021102706-appb-000007
也即x′＝x,y′＝y,z′＝z。
为了确定该第一变换矩阵，例如可以采用下述方式：基于目标雷达在所述第一安装姿态下的实际检测数据、以及预先确定的所述的目标雷达在所述预设标准姿态下的标准检测数据，确定所述第一变换矩阵。
示例性的，对目标雷达的标定过程，可以看作基于实际检测数据和标准检测数据对第一变换矩阵进行求解的过程。其中，实际检测数据包括了多个位置点分别在目标雷达获取的检测数据中的第一位置信息，也即上述(x,y,z)，且多个位置点预先标注了在基准坐标系中的第二位置信息，也即(x′,y′,z′)，则基于多个位置点分别对应的第一位置信息和第二位置信息，对第一变换矩阵进行联合求解，最终得到目标雷达在位于第一安装姿态下的第一变换矩阵。
在得到第一变换矩阵后，能够基于该变换矩阵，基于上述公式(1)对目标雷达在其第一安装姿态下的第一点云数据进行转换处理，得到在预设标准姿态下的第二点云数据。针对上述S103：
其中，在对第二点云数据执行检测任务时，例如可以利用神经网络对第二点云数据执行检测任务。其中，神经网络利用在预设标准姿态下的训练样本训练得到，其中，预设标准姿态下的训练样本例如可以以下述方式获得：
利用样本雷达获取在所述样本雷达的第二安装姿态下的第三点云数据；基于所述样本雷达的第二变换矩阵，将所述第三点云数据转换为在所述预设标准姿态下的第四点云数据；将所述第四点云数据作为所述训练样本。
此处，样本雷达的第二变换矩阵与目标雷达的第一变换矩阵的获取方式类似，在此不再赘述。
类似的，第二安装姿态满足以下中的至少一个：第二安装姿态对应的第二安装角度 与所述预设标准姿态对应的标准角度之间的第二角度差小于预设的角度差阈值；第二安装姿态对应的检测中心与所述预设标准姿态对应的检测中心之间的第二距离小于预设的距离阈值。
在确定了样本雷达的第二变换矩阵后，控制智能行驶装置行驶一定距离，并在智能行驶装置行驶过程中，采集第三点云数据；然后基于该第二变换矩阵，将第三点云数据变换为在预设标准姿态下的第四点云数据，并基于该第四点云数据对待训练的神经网络进行训练。
此处，在一种可能的实施方式中，若对神经网络进行训练的过程为有监督训练，则在得到第四点云数据后，还可以对第四点云数据进行标注，得到第四点云数据的标注信息；该标注信息用于在对待训练的神经网络进行训练的过程中，获得神经网络的损失，然后基于损失调整待训练的神经网络的参数。经过对待训练的神经网络的参数的多轮调整，得到预先训练的神经网络。
在另一种可能的实施方式中，若对第三点云数据进行了标注，得到第三点云数据的标注信息，在将第三点云数据转换为第四点云数据的时候，也会响应将第三点云数据的标注信息，转换为与第四点云数据适配的标注信息。
参见图2所示，在本公开另一实施例中，还提供另一种对第二点云数据执行检测任务，得到目标检测结果的具体方法，包括：
S201：基于所述第二点云数据，生成所述第二点云数据对应的至少一个稀疏矩阵；所述稀疏矩阵用于表征所述目标场景的不同位置处是否存在目标对象；
S202：基于所述至少一个稀疏矩阵、和所述第二点云数据，确定所述目标场景中包括的目标对象的三维检测数据；
S203：将所述三维检测数据作为所述目标检测结果。
在该实施例中，可以为获取到的第二点云数据生成对应的至少一个稀疏矩阵，该稀疏矩阵用于表征目标场景的不同位置处是否具有目标对象；这样，在基于稀疏矩阵和第二点云数据，确定目标对象的三维检测数据时，可以基于稀疏矩阵，确定存在对应的目标对象的目标位置，从而可以将与该目标位置对应的特征进行处理，而将不同位置中除目标位置之外的其他位置对应的特征不进行处理，这样就减少了得到目标对象的三维检测数据的计算量，提高了检测效率。
针对上述S201，在获取到目标场景的第二点云数据之后，可以基于第二点云数据，生成第二点云数据对应的至少一个稀疏矩阵。其中，稀疏矩阵可以表征目标场景的不同位置处是否具有目标对象。
这里，该稀疏矩阵可以为包括0和1的矩阵，即该稀疏矩阵为由0或1作为矩阵元素值构成得到的。比如，可以将目标场景中存在目标对象的位置处对应的矩阵元素值设置为1，可以将目标场景中不存在目标对象的位置处对应的矩阵元素值设置为0。
一种可选实施方式中，基于第二点云数据，生成第二点云数据对应的至少一个稀疏矩阵，可以包括：基于第二点云数据，确定用于检测目标对象的神经网络中，每一层卷积模块所对应的稀疏矩阵。
这里，该神经网络中可以包括多层卷积模块，每层卷积模块中可以包括一层卷积层，具体实施时，可以为每层卷积模块确定对应的稀疏矩阵，即为每层卷积层均确定对应的稀疏矩阵；或者，该神经网络中可以包括多个网络模块(block)，每个网络模块中包括多层卷积层，具体实施时，可以为每个网络模块确定对应的稀疏矩阵，即为网络模块中包括的多层卷积层确定一个对应的稀疏矩阵。其中，用于检测目标对象的神经网络的结构可以根据需要进行设置，此处仅为示例性说明。
在确定已训练的用于检测目标对象的神经网络之后，可以基于第二点云数据，为神经网络中的每一层卷积模块确定对应的稀疏矩阵。
上述实施方式下，可以基于第二点云数据，为神经网络的每一层卷积模块确定 对应的稀疏矩阵，使得每一层卷积模块可以基于稀疏矩阵对输入的特征图进行处理。
一种可选实施方式中，参见图3所示，基于第二点云数据，确定神经网络中每一层卷积模块对应的稀疏矩阵，可以包括：
S301：基于第二点云数据，生成初始稀疏矩阵。
此处，作为一可选实施方式，基于第二点云数据，生成初始稀疏矩阵，包括：
A1，确定第二点云数据对应的目标区域，并按照预设的栅格数量，将目标区域划分为多个栅格区域。
A2，基于第二点云数据对应的点所处的栅格区域，确定每个栅格区域对应的矩阵元素值。
A3，基于每个栅格区域对应的矩阵元素值，生成第二点云数据对应的初始稀疏矩阵。
这里，可以基于第二点云数据，判断每个栅格区域中是否存在第二点云数据对应的点，基于判断结果，确定每个栅格区域的矩阵元素值，比如，若栅格区域中存在第二点云数据对应的点，则该栅格区域的矩阵元素值为1，表征该栅格区域位置处存在目标对象，进而基于各个栅格区域对应的矩阵元素值，生成了初始稀疏矩阵，为后续确定目标对象的三维检测数据提供了数据支持。
示例性的，第二点云数据对应的目标区域可以为：基于激光雷达装置获取第二点云数据时的位置(例如，以该位置为起始位置)以及激光雷达装置能够探测的最远距离(例如，以该最远距离为长度)，确定得到的探测区域。其中，目标区域可以根据实际情况结合第二点云数据进行确定。
具体实施时，预设的栅格数量可以为N×M个，则可以将目标区域划分为N×M个栅格区域，N和M为正整数。其中，N和M的值可以根据实际需要进行设置。
第二点云数据中包括多个点的位置信息，可以基于每个点的位置信息，确定每个点所处的栅格区域，进而，可以针对每个栅格区域，在该栅格区域中存在对应的点时，则该栅格区域对应的矩阵元素值的可以为1；在该栅格区域中不存在对应的点时，则该栅格区域对应的矩阵元素值可以为0，因此确定了每个栅格区域对应的矩阵元素值。
在确定了每个栅格区域对应的矩阵元素值之后，可以基于每个栅格区域对应的矩阵元素值，生成第二点云数据对应的初始稀疏矩阵，其中，该初始稀疏矩阵的行数、列数与栅格数量对应，比如，若栅格数量为N×M个，则初始稀疏矩阵的行数为N，列数为M，即初始稀疏矩阵为N×M的矩阵。
参见图4所示，图中包括激光雷达装置41，以该激光雷达装置为中心，得到的目标区域42，并按照预设的栅格数量，将该目标区域划分为多个栅格区域，得到划分后的多个栅格区域421。再确定第二点云数据对应的多个点所处的栅格区域，将存在第二点云数据对应的点的栅格区域(即图中存在黑色阴影的栅格区域)的矩阵元素值设置为1，将不存在第二点云数据对应的点的栅格区域的矩阵元素值设置为0，得到了每个栅格区域的矩阵元素值。最后，基于每个栅格区域对应的矩阵元素值，生成第二点云数据对应的初始稀疏矩阵43。
承接上述S301，本公开实施例提供的基于第二点云数据，确定神经网络中每一层卷积模块对应的稀疏矩阵的方法还包括：S302：基于初始稀疏矩阵，确定与输入至神经网络的每一层卷积模块的特征图的目标尺寸匹配的稀疏矩阵。
上述实施方式中，可以基于第二点云数据，生成初始稀疏矩阵，再基于初始稀疏矩阵，为神经网络的每一层卷积模块确定对应的稀疏矩阵，且每一层卷积模块对应的稀疏矩阵与输入至该层卷积模块的特征图的目标尺寸相匹配，使得每一层卷积模块可以基于稀疏矩阵对输入的特征图进行处理。
在得到了初始稀疏矩阵之后，可以基于初始稀疏矩阵，确定与输入至神经网络的每一层卷积模块的特征图的目标尺寸匹配的稀疏矩阵。
作为一可选实施方式，可以通过下述方式，确定与输入至神经网络的每一层卷积模块的特征图的目标尺寸匹配的稀疏矩阵：
方式一、基于初始稀疏矩阵，确定神经网络中每一层卷积模块对应的输出稀疏矩阵，将该输出稀疏矩阵作为该层卷积模块对应的稀疏矩阵。
方式二、基于初始稀疏矩阵，确定神经网络中每一层卷积模块对应的输入稀疏矩阵，将该输入稀疏矩阵作为该层卷积模块对应的稀疏矩阵。
方式三、基于初始稀疏矩阵，确定神经网络中每一层卷积模块对应的输入稀疏矩阵和输出稀疏矩阵，将输入稀疏矩阵和输出稀疏矩阵进行融合，得到融合稀疏矩阵，将融合稀疏矩阵作为该层卷积模块对应的稀疏矩阵。
这里，稀疏矩阵可以由输出稀疏矩阵得到，也可以由输入稀疏矩阵得到，或者，还可以为由输入稀疏矩阵和输出稀疏矩阵融合得到。
上述实施方式中，设置多种方式，生成每一层卷积模块对应的稀疏矩阵，即稀疏矩阵可以为输入稀疏矩阵，也可以为输出稀疏矩阵，还可以为基于输入稀疏矩阵和输出稀疏矩阵生成的融合稀疏矩阵。
针对方式一，该方式是由输出稀疏矩阵得到稀疏矩阵。具体实施时，可以基于初始稀疏数据，确定神经网络中，每一层卷积模块对应的输出稀疏矩阵，该输出稀疏矩阵即为稀疏矩阵。其中，该输出稀疏矩阵可以用于表征神经网络每一层卷积模块的输出结果中对应目标场景的不同位置处是否具有对应的目标对象的三维检测数据。比如，若神经网络的每一层卷积模块的输出结果中，对应目标场景的位置A处具有对应的目标对象的三维检测数据时，则在稀疏矩阵中与该位置A对应的位置处的矩阵值可以为1；若位置A处不具有对应的目标对象的三维检测数据时，则在稀疏矩阵中，与该位置A对应的位置处的矩阵值可以为0。
针对方式二，该方式是由输入稀疏矩阵得到稀疏矩阵。具体实施时，可以基于初始稀疏数据，确定神经网络中，每一层卷积模块对应的输入稀疏矩阵，该输入稀疏矩阵即为稀疏矩阵。其中，输入稀疏矩阵可以为表征神经网络每一层卷积模块的输入数据中，对应目标场景的不同位置处是否具有对应的目标对象的三维检测数据。
针对方式三，可以通过方式一确定每一层卷积模块对应的输出稀疏矩阵，并通过方式二确定每一层卷积模块对应的输入稀疏矩阵，将每一层卷积模块对应的输入稀疏矩阵和输出稀疏矩阵进行融合，得到融合稀疏矩阵，将融合稀疏矩阵作为该层卷积模块对应的稀疏矩阵。
在具体实施时，可以将输入稀疏矩阵和输出稀疏矩阵取交集，得到融合稀疏矩阵；也可以将输入稀疏矩阵和输出稀疏矩阵取并集，得到融合稀疏矩阵。比如，若输入稀疏矩阵为：
Figure PCTCN2021102706-appb-000008
若输出稀疏矩阵为：
Figure PCTCN2021102706-appb-000009
则将输入稀疏矩阵和输出稀疏矩阵取交集，得到的融合稀疏矩阵为：
Figure PCTCN2021102706-appb-000010
则将输入稀疏矩阵和输出稀疏矩阵取并集，得到的融合稀疏矩阵为：
Figure PCTCN2021102706-appb-000011
一种可选实施方式中，基于初始稀疏矩阵，确定神经网络中，每一层卷积模块对应的输入稀疏矩阵，可以包括：
B1，将初始稀疏矩阵作为神经网络的第一层卷积模块对应的输入稀疏矩阵。
B2，基于第i-1层卷积模块对应的输入稀疏矩阵，确定第i层卷积模块对应的、与输入至第i层卷积模块的特征图的目标尺寸匹配的输入稀疏矩阵；其中，i为大于1、且小于n+1的正整数，n为神经网络的卷积模块的总层数。
这里，初始稀疏矩阵可以作为神经网络的第一层卷积模块对应的输入稀疏矩阵。第二层卷积模块对应的输入稀疏矩阵可以由第一层卷积模块对应的输入稀疏矩阵得到，且第二层卷积模块对应的输入稀疏矩阵的行数和列数与输入至第二层卷积模块的特征图的目标尺寸一致。
示例性的，可以利用图像膨胀处理操作或图像腐蚀处理操作，对第一层卷积模块对应的输入稀疏矩阵进行处理，得到处理后的稀疏矩阵，将处理后的稀疏矩阵的行列数调整为与输入至第二层卷积模块的特征图的目标尺寸匹配之后，得到第二层卷积模块的输入稀疏矩阵。依次类推，可以得到第一层卷积模块对应的输入稀疏矩阵、第二层卷积模块对应的输入稀疏矩阵、……、第n层卷积模块对应的输入稀疏矩阵(即神经网络最后一层卷积模块对应的输入稀疏矩阵)。
示例性的，可以预先确定膨胀处理范围，基于膨胀处理范围对输入稀疏矩阵进行图像膨胀处理，得到处理后的稀疏矩阵，其中，膨胀处理范围可以为基于目标对象的尺寸阈值确定的，也可以根据实际需要进行确定。比如，若输入稀疏矩阵为：
Figure PCTCN2021102706-appb-000012
则膨胀处理后的稀疏矩阵可以为：
Figure PCTCN2021102706-appb-000013
其中，上述膨胀处理过程仅为示例性说明。
示例性的，输入稀疏矩阵的腐蚀处理过程为膨胀处理过程的逆过程，具体的，可以预先确定腐蚀处理范围，基于腐蚀处理范围对输入稀疏矩阵进行图像腐蚀处理，得到处理后的稀疏矩阵。其中，腐蚀处理范围可以为基于目标对象的尺寸阈值确定的，也可以根据实际需要进行确定。比如，若输入稀疏矩阵为：
Figure PCTCN2021102706-appb-000014
则腐蚀处理后的稀疏矩阵可以为：
Figure PCTCN2021102706-appb-000015
其中，上述腐蚀处理过程仅为示例性说明。
在具体实施时，可以通过上采样或下采样的方式，将处理后的稀疏矩阵的行数和列数调整为与输入至第二层卷积模块的特征图的目标尺寸匹配的矩阵，得到第二层卷积模块的输入稀疏矩阵，其中，对处理后的稀疏矩阵的行数和列数进行调整的过程有多种，此处仅为示例性说明。
在具体实施时，还可以对稀疏矩阵的稀疏程度进行调整，比如，可以通过调整栅格的数量，对稀疏矩阵的稀疏程度进行调整；或者也可以通过腐蚀处理过程对稀疏矩阵的稀疏程度进行调整。其中，稀疏矩阵的稀疏程度为：稀疏矩阵中矩阵元素值为1的矩阵元素的数量与稀疏矩阵中包括的全部矩阵元素的总数的比值。
上述方式中，可以将初始稀疏矩阵作为第一层卷积模块对应的输入稀疏矩阵，并依次确定得到每一层卷积模块的输入稀疏矩阵，进而可以基于该输入稀疏矩阵确定稀疏矩阵，为后续基于每一层卷积模块的稀疏矩阵，确定目标对象的三维检测数据提供了数据支持。
一种可能的实施方式中，基于初始稀疏矩阵，确定神经网络中，每一层卷积模块对应的输出稀疏矩阵，可以包括：
C1，基于目标对象的尺寸阈值和初始稀疏矩阵，确定神经网络对应的输出稀疏矩阵。
C2，基于输出稀疏矩阵，生成第n层卷积模块对应的、与输入至第n层卷积模块的特征图的目标尺寸匹配的输出稀疏矩阵。
C3，基于第j+1层卷积模块对应的输出稀疏矩阵，生成第j层卷积模块对应的、与输入至第j层卷积模块的特征图的目标尺寸匹配的输出稀疏矩阵，其中，j为大于等于1、且小于n的正整数，n为神经网络的卷积模块的总层数。
这里，可以先根据目标对象的尺寸阈值，确定膨胀处理范围，基于膨胀处理范围对初始稀疏矩阵进行膨胀处理，得到处理后的稀疏矩阵，该处理后的稀疏矩阵即为神经网络对应的输出稀疏矩阵。其中，膨胀处理过程可参考上述描述，此处不再进行赘述。
利用输出稀疏矩阵，确定神经网络第n层卷积模块(即神经网络的最后一层卷积模块)的输出稀疏矩阵，依次类推，得到第n-1层卷积模块的输出稀疏矩阵、……、第二层卷积模块的输出稀疏矩阵、第一层卷积模块的输出稀疏矩阵。
示例性的，可以利用图像膨胀处理操作或图像腐蚀处理操作，对前一层卷积模块对应的输出稀疏矩阵进行处理，得到处理后的稀疏矩阵，将处理后的稀疏矩阵的行数和列数调整为与输入至当前层卷积模块的特征图的目标尺寸匹配之后，得到当前层卷积 模块的输出稀疏矩阵。其中，确定每一层卷积模块的输出稀疏矩阵的过程，可参考上述确定输入稀疏矩阵的过程，此处不再进行详细说明。
对于由输入稀疏矩阵和输出稀疏矩阵的融合得到神经网络的每一层卷积模块的目标稀疏矩阵的情况，可以分别利用上述方法得到每一层卷积模块的输出稀疏矩阵和输入稀疏矩阵，将得到的输出稀疏矩阵和输入稀疏矩阵进行融合，得到每一卷积模块的稀疏矩阵。
上述方式中，可以基于初始稀疏矩阵，确定输出稀疏矩阵，利用输出稀疏矩阵依次确定第n层卷积模块的输出稀疏矩阵、…、第一层卷积模块的输出稀疏矩阵，进而可以基于每一层的输出稀疏矩阵确定稀疏矩阵，为后续基于每一层卷积模块的稀疏矩阵，确定目标对象的三维检测数据提供了数据支持。
针对S202：在具体实施时，可以基于至少一个稀疏矩阵、第二点云数据、和用于检测目标对象的神经网络，确定目标场景中包括的目标对象的三维检测数据。该三维检测数据包括目标对象的检测框的中心点的坐标、检测框的三维尺寸、检测框的朝向角、目标对象的类别、时间戳、以及检测框的置信度。
这里，目标对象的三维检测框的位置不能超出目标区域的位置，即若三维检测框的中心点坐标为(X，Y，Z)，尺寸为长L、宽W、高H时，则满足以下条件：0≤X-2/L，X+2/L<N max，0≤Y-W/2，Y+W/2<M max，其中，N max和M max是目标区域的长度阈值和宽度阈值。
一种可选实施方式中，基于至少一个稀疏矩阵、和第二点云数据，确定目标场景中包括的目标对象的三维检测数据，包括：
步骤一、基于第二点云数据，生成第二点云数据对应的目标点云特征图。
步骤二、基于目标点云特征图和至少一个稀疏矩阵，利用检测目标对象的神经网络，确定目标场景中包括的目标对象的三维检测数据，其中，神经网络中包括多层卷积模块。
在具体实施时，可以将第二点云数据输入至神经网络中，对第二点云数据进行预处理，生成第二点云数据对应的目标点云特征图，再利用目标点云特征图、至少一个稀疏矩阵、和神经网络，确定目标场景中包括的目标对象的三维检测数据。
步骤一中，基于第二点云数据，生成第二点云数据对应的目标点云特征图，可以包括：
针对每个栅格区域，基于位于栅格区域内的所述第二点云数据所指示的点的坐标信息，确定栅格区域对应的特征信息；其中，栅格区域为按照预设的栅格数量，将第二点云数据对应的目标区域划分生成的。
基于每个栅格区域对应的特征信息，生成第二点云数据对应的目标点云特征图。
针对每个栅格区域，若该栅格区域中存在第二点云数据对应的点时，则将该栅格区域内第二点云数据所指示的各个点的坐标信息，构成该栅格区域对应的特征信息；若该栅格区域中不存在第二点云数据对应的点时，则该栅格区域的特征信息可以为0。
基于每个栅格区域对应的特征信息，生成了第二点云数据对应的目标点云特征图。其中，目标点云特征图的尺寸可以为N×M×C，目标点云特征图的尺寸N×M与第一层卷积模块的稀疏矩阵的尺寸相一致，目标点云特征图的C可以为各个栅格区域中包括的点的数量最大值，比如，若各个栅格区域中栅格区域A中包括的点的数量最多，比如，栅格区域中包括50个点，则C的值为50，即目标点云特征图中包括50个尺寸为N×M的特征图，每个特征图中包括至少一个点的坐标信息。
在上述实施方式下，基于每个栅格区域对应的特征信息生成了第二点云数据对应的目标点云特征图，目标点云特征图包括每个点的位置信息，进而基于目标点云特征图和至少一个稀疏矩阵，可以较准确的确定目标场景中包括的目标对象的三维检测数据。
步骤二中，可以基于目标点云特征图、至少一个稀疏矩阵、和神经网路，确定 目标场景中包括的目标对象的三维检测数据。
具体实施时，可以通过下述两种方式，确定目标场景中包括的目标对象的三维检测数据：
方式一：基于目标点云特征图和至少一个稀疏矩阵，确定目标场景中包括的目标对象的三维检测数据，包括：
一、基于神经网络中第一层卷积模块对应的稀疏矩阵，确定目标点云特征图中的待卷积特征信息，利用第一层卷积模块，对目标点云特征图中的待卷积特征信息进行卷积处理，生成输入至第二层卷积模块的特征图。
二、基于神经网络中第k层卷积模块对应的稀疏矩阵，确定输入至第k层卷积模块的目标点云特征图中的待卷积特征信息，利用神经网络的第k层卷积模块，对第k层卷积模块的目标点云特征图中的待卷积特征信息进行卷积处理，生成输入至第k+1层卷积模块的特征图，k为大于1、小于n的正整数，n为神经网络的卷积模块的层数。
三、基于神经网络中第n层卷积模块对应的稀疏矩阵，确定输入至第n层卷积模块的目标点云特征图中的待卷积特征信息，利用神经网络的第n层卷积模块，对第n层卷积模块的目标点云特征图中的待卷积特征信息进行卷积处理，得到目标场景中包括的目标对象的三维检测数据。
上述实施方式中，第一层卷积模块中，可以利用第一层卷积模块的稀疏矩阵，确定输入至第一层卷积模块中的目标点云特征图中的待卷积特征信息。具体的，可以确定稀疏矩阵中矩阵值为1的目标位置，将目标点云特征图中与目标位置对应的位置的特征信息，确定为待卷积特征信息。
进而利用第一层卷积模块，对目标点云特征图中的待卷积特征信息进行卷积处理，生成输入至第二层卷积模块的特征图。在接着利用第二层卷积模块的稀疏矩阵，确定输入至第二层卷积模块的特征图中的待卷积信息，并利用第二层卷积模块对第二层卷积模块的特征图中的待卷积特征信息进行卷积处理，生成输入至第三层卷积模块的特征图，依次类推，得到输入至第n层卷积模块(神经网络中最后一层卷积模块)的特征图，通过确定第n层卷积模块的待卷积信息，并对第n层卷积模块的待卷积信息进行卷积处理，得到目标场景中包括的目标对象的三维检测数据。
这里，可以基于每一层卷积模块的稀疏矩阵和输入的特征图，确定待卷积特征信息，对待卷积特征信息进行卷积处理，而对特征图中除待卷积特征信息之外的其他特征信息不进行卷积处理，减少了每一层卷积模块进行卷积处理的计算量，提高了每一层卷积模块的运算效率，进而可以减少神经网络的运算量，提高目标对象的检测效率。
方式二，基于目标点云特征图和至少一个稀疏矩阵，确定目标场景中包括的目标对象的三维检测数据，包括：
一、针对神经网络中除最后一层卷积模块之外的其他每一层卷积模块，基于该层卷积模块对应的稀疏矩阵和输入至该层卷积模块的特征图，确定该层卷积模块对应的卷积向量；基于该层卷积模块对应的卷积向量，确定输入至下一层卷积模块的特征图。
二、基于最后一层卷积模块对应的稀疏矩阵和输入至最后一层卷积模块的特征图，确定最后一层卷积模块对应的卷积向量；基于最后一层卷积模块对应的卷积向量，确定目标场景中包括的目标对象的三维检测数据。
上述实施方式中，还可以基于每一层卷积模块对应的目标输入矩阵和输入至该层卷积模块的特征图，确定该层卷积模块对应的卷积向量。比如，针对第一层卷积模块，可以确定第一层卷积模块的稀疏矩阵中矩阵值为1的目标位置，并确定目标点云特征图中与目标位置对应的位置的特征信息，提取与目标位置对应的特征信息，构成了第一层卷积模块对应的卷积向量。
进一步的，可以利用img2col和col2img技术，通过第一层卷积模块对对应的卷积向量进行矩阵乘法运算，得到输入至第二卷积模块的特征图。基于相同的处理过程， 可以得到输入至最后一层卷积模块的特征图，基于最后一层卷积模块对应的稀疏矩阵和特征图，确定最后一层卷积模块对应的卷积向量，对最后一层卷积模块对应的卷积向量进行处理，确定目标场景中包括的目标对象的三维检测数据。
这里，可以基于每一层卷积模块的稀疏矩阵和输入的特征图，生成每一层卷积模块对应的卷积向量，该卷积向量中包括特征图中的待处理的特征信息，该待处理的特征信息为：与稀疏矩阵中指示的存在目标对象的三维检测数据的位置匹配的、特征图中的特征信息，对生成的卷积向量进行处理，而对特征图中除待处理的特征信息之外的其他特征信息不进行处理，减少了每一层卷积模块进行卷积处理的计算量，提高了每一层卷积模块的运算效率，进而可以减少神经网络的运算量，提高目标对象的检测效率。
参见图5所示，本公开实施例还提供一种智能行驶控制方法，包括：
S501：利用设置在行驶装置上的目标雷达采集点云数据；
S502：基于本公开任一实施例所述的点云数据的处理方法对所述点云数据进行检测处理，得到检测结果；
S503：基于所述检测结果，控制所述行驶装置。
本公开实施例提供的点云数据的处理方法，由于采用的神经网络的复杂度更低，对计算资源的需求量较少，因此更易于部署在终端设备中，且具有更高的检测精度。
在具体实施中，行驶装置例如但不限于下述任一种：自动驾驶车辆、装有高级驾驶辅助系统(Advanced Driving Assistance System，ADAS)的车辆、机器人等。
控制行驶装置，例如包括控制行驶装置加速、减速、转向、制动等，或者可以播放语音提示信息，以提示驾驶员控制行驶装置加速、减速、转向、制动等。
本领域技术人员可以理解，在具体实施方式的上述方法中，各步骤的撰写顺序并不意味着严格的执行顺序而对实施过程构成任何限定，各步骤的具体执行顺序应当以其功能和可能的内在逻辑确定。
基于同一发明构思，本公开实施例中还提供了与点云数据的处理方法对应的点云数据的处理装置，由于本公开实施例中的装置解决问题的原理与本公开实施例上述点云数据的处理方法相似，因此装置的实施可以参见方法的实施，重复之处不再赘述。
参照图6所示，本公开实施例提供一种点云数据的处理装置，包括获取模块61、转换模块62和检测模块63。获取模块61，用于获取目标雷达在第一安装姿态下采集目标场景得到的第一点云数据。转换模块62，用于基于预先确定的第一变换矩阵，将所述第一点云数据转换为预设标准姿态下的第二点云数据。检测模块63，对所述第二点云数据执行检测任务，得到目标检测结果。
一种可能的实施方式中，所述第一安装姿态满足以下中的至少一个：所述目标雷达在第一安装姿态时的第一安装角度与所述目标雷达在预设标准姿态时的标准角度之间的角度差小于预设的角度差阈值；目标雷达在第一安装姿态下的检测中心与目标雷达在所述预设标准姿态下的检测中心之间的距离小于预设的距离阈值。
一种可能的实施方式中，所述点云数据的处理装置还包括：第一确定模块64，用于采用下述方式确定所述角度差阈值和所述距离阈值中的至少一个：基于对多个验证样本集分别进行检测所得到的检测结果，确定所述角度差阈值和所述距离阈值中的至少一个，其中，每个验证样本集中包括多组验证数据；用于采集不同验证样本集的雷达具有不同的安装姿态。
一种可能的实施方式中，所述点云数据的处理装置还包括：第二确定模块65，用于采用下述方式确定所述第一变换矩阵：基于目标雷达在所述第一安装姿态下的实际检测数据、以及预先确定的所述的目标雷达在所述预设标准姿态下的标准检测数据，确定所述第一变换矩阵。
一种可能的实施方式中，所述检测模块63，在对所述第二点云数据执行检测任务，得到目标检测结果时，用于：基于所述第二点云数据，生成所述第二点云数据对应 的至少一个目标的稀疏矩阵；所述稀疏矩阵用于表征所述目标场景的不同位置处是否存在目标对象；基于所述至少一个稀疏矩阵、和所述第二点云数据，确定所述目标场景中包括的目标对象的三维检测数据；将所述三维检测数据作为所述目标检测结果。
一种可能的实施方式中，所述检测模块63，在基于所述第二点云数据，生成所述第二点云数据对应的至少一个稀疏矩阵时，用于：基于所述第二点云数据，确定用于检测所述目标对象的神经网络中每一层卷积模块所对应的稀疏矩阵。
一种可能的实施方式中，所述检测模块63，在基于所述第二点云数据，确定用于检测所述目标对象的神经网络中每一层卷积模块所对应的稀疏矩阵时，用于：基于所述第二点云数据，生成初始稀疏矩阵；基于所述初始稀疏矩阵，确定与输入至所述神经网络的每一层卷积模块的特征图的目标尺寸匹配的稀疏矩阵。
一种可能的实施方式中，所述检测模块63，在基于所述第二点云数据，生成初始稀疏矩阵时，用于：确定所述第二点云数据对应的目标区域，并按照预设的栅格数量，将所述目标区域划分为多个栅格区域；基于所述第二点云数据对应的点所处的栅格区域，确定每个栅格区域对应的矩阵元素值；基于每个栅格区域对应的矩阵元素值，生成所述第二点云数据对应的初始稀疏矩阵。
一种可能的实施方式中，所述检测模块63，在基于所述初始稀疏矩阵，确定与输入至所述神经网络的每一层卷积模块的特征图的目标尺寸匹配的稀疏矩阵时，用于以下任一：基于所述初始稀疏矩阵，确定所述神经网络中每一层卷积模块对应的输出稀疏矩阵，将该输出稀疏矩阵作为该层卷积模块对应的所述稀疏矩阵；基于所述初始稀疏矩阵，确定所述神经网络中每一层卷积模块对应的输入稀疏矩阵，将该输入稀疏矩阵作为该层卷积模块对应的所述稀疏矩阵；基于所述初始稀疏矩阵，确定所述神经网络中每一层卷积模块对应的输入稀疏矩阵和输出稀疏矩阵，将所述输入稀疏矩阵和输出稀疏矩阵进行融合，得到融合稀疏矩阵，将所述融合稀疏矩阵作为该层卷积模块对应的稀疏矩阵。
一种可能的实施方式中，所述检测模块63，在基于所述初始稀疏矩阵，确定所述神经网络中每一层卷积模块对应的输入稀疏矩阵是，用于：将所述初始稀疏矩阵作为所述神经网络的第一层卷积模块对应的输入稀疏矩阵；基于第i-1层卷积模块对应的输入稀疏矩阵，确定第i层卷积模块对应的、与输入至所述第i层卷积模块的特征图的目标尺寸匹配的输入稀疏矩阵；其中，i为大于1、且小于n+1的正整数，n为所述神经网络的卷积模块的总层数。
一种可能的实施方式中，所述检测模块63，在基于所述初始稀疏矩阵，确定所述神经网络中每一层卷积模块对应的输出稀疏矩阵时，用于：基于所述目标对象的尺寸阈值和所述初始稀疏矩阵，确定所述神经网络对应的输出稀疏矩阵；基于所述输出稀疏矩阵，生成第n层卷积模块对应的、与输入至所述第n层卷积模块的特征图的目标尺寸匹配的输出稀疏矩阵；基于第j+1层卷积模块对应的输出稀疏矩阵，生成第j层卷积模块对应的、与输入至所述第j层卷积模块的特征图的目标尺寸匹配的输出稀疏矩阵，其中，j为大于等于1、且小于n的正整数，n为所述神经网络的卷积模块的总层数。
一种可能的实施方式中，所述检测模块63，在基于所述至少一个稀疏矩阵、和所述第二点云数据，确定所述目标场景中包括的目标对象的三维检测数据时，用于：基于所述第二点云数据，生成所述第二点云数据对应的点云特征图；基于所述点云特征图和所述至少一个稀疏矩阵，利用检测目标对象的神经网络，确定所述目标场景中包括的目标对象的三维检测数据，其中，所述神经网络中包括多层卷积模块。
一种可能的实施方式中，所述检测模块63，在基于所述第二点云数据，生成所述第二点云数据对应的点云特征图时，用于：针对每个栅格区域，基于位于所述栅格区域内第二点云数据所指示的点的坐标信息，确定所述栅格区域对应的特征信息；其中，所述栅格区域为按照预设的栅格数量，将所述第二点云数据对应的目标区域划分生成的；基于每个栅格区域对应的特征信息，生成所述第二点云数据对应的点云特征图。
一种可能的实施方式中，所述检测模块63在基于所述点云特征图和所述至少一个稀疏矩阵，利用检测目标对象的神经网络，确定所述目标场景中包括的目标对象的三维检测数据时，用于：基于所述神经网络中第一层卷积模块对应的稀疏矩阵，确定所述点云特征图中的待卷积特征信息，利用所述第一层卷积模块，对所述点云特征图中的所述待卷积特征信息进行卷积处理，生成输入至第二层卷积模块的特征图；基于所述神经网络中第k层卷积模块对应的稀疏矩阵，确定输入至所述第k层卷积模块的特征图中的待卷积特征信息，利用所述神经网络的第k层卷积模块，对所述第k层卷积模块的特征图中的待卷积特征信息进行卷积处理，生成输入至第k+1层卷积模块的特征图，k为大于1、小于n的正整数，n为所述神经网络的卷积模块的总层数；基于所述神经网络中第n层卷积模块对应的稀疏矩阵，确定输入至所述第n层卷积模块的特征图中的待卷积特征信息，利用所述神经网络的第n层卷积模块，对所述第n层卷积模块的特征图中的待卷积特征信息进行卷积处理，得到所述目标场景中包括的目标对象的三维检测数据。
一种可能的实施方式中，所述检测模块63，在基于所述点云特征图和所述至少一个稀疏矩阵，利用检测目标对象的神经网络，确定所述目标场景中包括的目标对象的三维检测数据时，用于：针对所述神经网络中除最后一层卷积模块之外的其他每一层卷积模块，基于该层卷积模块对应的稀疏矩阵和输入至该层卷积模块的特征图，确定该层卷积模块对应的卷积向量；基于该层卷积模块对应的所述卷积向量，确定输入至下一层卷积模块的特征图；基于最后一层卷积模块对应的稀疏矩阵和输入至最后一层卷积模块的特征图，确定最后一层卷积模块对应的卷积向量；基于最后一层卷积模块对应的所述卷积向量，确定所述目标场景中包括的目标对象的三维检测数据。
参见图7所示，本公开实施例还提供一种智能行驶控制装置，包括：
获取模块71，用于获取设置在行驶装置上的目标雷达采集的点云数据；
处理模块72，用于基于本公开任一实施例所述的点云数据的处理方法对所述点云数据进行检测处理，得到检测结果；
控制模块73，用于基于所述检测结果，控制所述行驶装置。
关于装置中的各模块的处理流程、以及各模块之间的交互流程的描述可以参照上述方法实施例中的相关说明，这里不再详述。
如图8所示，本公开实施例还提供了一种计算机设备，包括处理器11和存储器12；所述存储器12存储有所述处理器11可执行的机器可读指令，当计算机设备运行时，所述机器可读指令被所述处理器执行以实现下述步骤：
获取目标雷达在第一安装姿态下采集目标场景得到的第一点云数据；
基于预先确定的第一变换矩阵，将所述第一点云数据转换为预设标准姿态下的第二点云数据；
对所述第二点云数据执行检测任务，得到目标检测结果。
或者，所述机器可读指令被所述处理器执行以实现下述步骤：
利用设置在行驶装置上的目标雷达采集的点云数据；
基于本公开实施例任一项所述的点云数据的处理方法对所述点云数据进行检测处理，得到检测结果；
基于所述检测结果，控制所述行驶装置。
上述指令的具体执行过程可以参考本公开实施例中所述的点云数据的处理方法或智能行驶控制方法的步骤，此处不再赘述。
本公开实施例还提供一种计算机可读存储介质，该计算机可读存储介质上存储有计算机程序，该计算机程序被处理器运行时执行上述方法实施例中所述的点云数据的处理方法或智能行驶控制方法。其中，该存储介质可以是易失性或非易失的计算机可读取存储介质。
本公开实施例所提供的点云数据的处理方法或智能行驶控制方法的计算机程序 产品，包括存储了程序代码的计算机可读存储介质，所述程序代码包括的指令可用于执行上述方法实施例中所述的点云数据的处理方法或智能行驶控制方法的步骤，具体可参见上述方法实施例，在此不再赘述。
本公开实施例还提供一种计算机程序，该计算机程序被处理器执行时实现前述实施例的任意一种方法。该计算机程序产品可以具体通过硬件、软件或其结合的方式实现。在一个可选实施例中，所述计算机程序产品具体体现为计算机存储介质，在另一个可选实施例中，计算机程序产品具体体现为软件产品，例如软件开发包(Software Development Kit，SDK)等等。
所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的系统和装置的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。在本公开所提供的几个实施例中，应该理解到，所揭露的系统、装置和方法，可以通过其它的方式实现。以上所描述的装置实施例仅仅是示意性的，例如，所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，又例如，多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些通信接口，装置或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外，在本公开各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。
所述功能如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个处理器可执行的非易失的计算机可读取存储介质中。基于这样的理解，本公开的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本公开各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(Read-Only Memory，ROM)、随机存取存储器(Random Access Memory，RAM)、磁碟或者光盘等各种可以存储程序代码的介质。
最后应说明的是：以上所述实施例，仅为本公开的具体实施方式，用以说明本公开的技术方案，而非对其限制，本公开的保护范围并不局限于此，尽管参照前述实施例对本公开进行了详细的说明，本领域的普通技术人员应当理解：任何熟悉本技术领域的技术人员在本公开揭露的技术范围内，其依然可以对前述实施例所记载的技术方案进行修改或可轻易想到变化，或者对其中部分技术特征进行等同替换；而这些修改、变化或者替换，并不使相应技术方案的本质脱离本公开实施例技术方案的精神和范围，都应涵盖在本公开的保护范围之内。因此，本公开的保护范围应所述以权利要求的保护范围为准。

Claims (20)

  1. 一种点云数据的处理方法，包括：
    获取目标雷达在第一安装姿态下采集目标场景得到的第一点云数据；
    基于预先确定的第一变换矩阵，将所述第一点云数据转换为预设标准姿态下的第二点云数据；
    对所述第二点云数据执行检测任务，得到目标检测结果。
  2. 根据权利要求1所述的处理方法，其特征在于，所述第一安装姿态满足以下至少一个：
    所述目标雷达在所述第一安装姿态时的第一安装角度与所述目标雷达在所述预设标准姿态时的标准角度之间的角度差小于预设的角度差阈值；
    所述目标雷达在所述第一安装姿态下的检测中心与所述目标雷达在所述预设标准姿态下的检测中心之间的距离小于预设的距离阈值。
  3. 根据权利要求2所述的处理方法，其特征在于，采用下述方式确定所述角度差阈值和所述距离阈值中的至少一个：
    基于对多个验证样本集分别进行检测所得到的检测结果，确定所述角度差阈值和所述距离阈值中的至少一个，其中，每个验证样本集中包括多组验证数据；用于采集不同验证样本集的雷达具有不同的安装姿态。
  4. 根据权利要求1至3任一项所述的处理方法，其特征在于，采用下述方式确定所述第一变换矩阵：
    基于所述目标雷达在所述第一安装姿态下的实际检测数据、以及预先确定的所述目标雷达在所述预设标准姿态下的标准检测数据，确定所述第一变换矩阵。
  5. 根据权利要求1至4任一项所述的处理方法，其特征在于，所述对所述第二点云数据执行检测任务，得到目标检测结果，包括：
    基于所述第二点云数据，生成所述第二点云数据对应的至少一个稀疏矩阵；所述稀疏矩阵用于表征所述目标场景的不同位置处是否存在目标对象；
    基于所述至少一个稀疏矩阵、和所述第二点云数据，确定所述目标场景中包括的目标对象的三维检测数据；
    将所述三维检测数据作为所述目标检测结果。
  6. 根据权利要求5所述的处理方法，其特征在于，所述基于所述第二点云数据，生成所述第二点云数据对应的至少一个稀疏矩阵，包括：
    基于所述第二点云数据，确定用于检测所述目标对象的神经网络中每一层卷积模块所对应的稀疏矩阵。
  7. 根据权利要求6所述的处理方法，其特征在于，所述基于所述第二点云数据，确定用于检测所述目标对象的神经网络中每一层卷积模块所对应的稀疏矩阵，包括：
    基于所述第二点云数据，生成初始稀疏矩阵；
    基于所述初始稀疏矩阵，确定与输入至所述神经网络的每一层卷积模块的特征图的目标尺寸匹配的稀疏矩阵。
  8. 根据权利要求7所述的处理方法，其特征在于，所述基于所述第二点云数据，生成初始稀疏矩阵，包括：
    确定所述第二点云数据对应的目标区域；
    按照预设的栅格数量，将所述目标区域划分为多个栅格区域；
    基于所述第二点云数据对应的点所处的栅格区域，确定每个栅格区域对应的矩阵元素值；
    基于每个栅格区域对应的矩阵元素值，生成所述第二点云数据对应的初始稀疏矩阵。
  9. 根据权利要求7或8所述的处理方法，其特征在于，所述基于所述初始稀疏矩阵，确定与输入至所述神经网络的每一层卷积模块的特征图的目标尺寸匹配的稀疏矩阵， 包括以下任一：
    基于所述初始稀疏矩阵，确定所述神经网络中每一层卷积模块对应的输出稀疏矩阵，将该输出稀疏矩阵作为该层卷积模块对应的所述稀疏矩阵；
    基于所述初始稀疏矩阵，确定所述神经网络中每一层卷积模块对应的输入稀疏矩阵，将该输入稀疏矩阵作为该层卷积模块对应的所述稀疏矩阵；
    基于所述初始稀疏矩阵，确定所述神经网络中每一层卷积模块对应的输入稀疏矩阵和输出稀疏矩阵，将所述输入稀疏矩阵和输出稀疏矩阵进行融合，得到融合稀疏矩阵，将所述融合稀疏矩阵作为该层卷积模块对应的所述稀疏矩阵。
  10. 根据权利要求9所述的处理方法，其特征在于，所述基于所述初始稀疏矩阵，确定所述神经网络中每一层卷积模块对应的输入稀疏矩阵，包括：
    将所述初始稀疏矩阵作为所述神经网络的第一层卷积模块对应的输入稀疏矩阵；
    基于第i-1层卷积模块对应的输入稀疏矩阵，确定第i层卷积模块对应的、与输入至所述第i层卷积模块的特征图的目标尺寸匹配的输入稀疏矩阵；其中，i为大于1、且小于n+1的正整数，n为所述神经网络的卷积模块的总层数。
  11. 根据权利要求9所述的处理方法，其特征在于，所述基于所述初始稀疏矩阵，确定所述神经网络中每一层卷积模块对应的输出稀疏矩阵，包括：
    基于所述目标对象的尺寸阈值和所述初始稀疏矩阵，确定所述神经网络对应的输出稀疏矩阵；
    基于所述输出稀疏矩阵，生成第n层卷积模块对应的、与输入至所述第n层卷积模块的特征图的目标尺寸匹配的输出稀疏矩阵；
    基于第j+1层卷积模块对应的输出稀疏矩阵，生成第j层卷积模块对应的、与输入至所述第j层卷积模块的特征图的目标尺寸匹配的输出稀疏矩阵，其中，j为大于等于1、且小于n的正整数，n为所述神经网络的卷积模块的总层数。
  12. 根据权利要求5至11任一项所述的处理方法，其特征在于，所述基于所述至少一个稀疏矩阵、和所述第二点云数据，确定所述目标场景中包括的目标对象的三维检测数据，包括：
    基于所述第二点云数据，生成所述第二点云数据对应的点云特征图；
    基于所述点云特征图和所述至少一个稀疏矩阵，利用检测目标对象的神经网络，确定所述目标场景中包括的目标对象的三维检测数据，其中，所述神经网络中包括多层卷积模块。
  13. 根据权利要求12所述的处理方法，其特征在于，所述基于所述第二点云数据，生成所述第二点云数据对应的点云特征图，包括：
    针对每个栅格区域，基于位于所述栅格区域内的所述第二点云数据所指示的点的坐标信息，确定所述栅格区域对应的特征信息；其中，所述栅格区域为按照预设的栅格数量，将所述第二点云数据对应的目标区域划分生成的；
    基于每个所述栅格区域对应的特征信息，生成所述第二点云数据对应的点云特征图。
  14. 根据权利要求12或13所述的处理方法，其特征在于，所述基于所述点云特征图和所述至少一个稀疏矩阵，利用检测目标对象的神经网络，确定所述目标场景中包括的目标对象的三维检测数据，包括：
    基于所述神经网络中第一层卷积模块对应的稀疏矩阵，确定所述点云特征图中的待卷积特征信息，利用所述第一层卷积模块，对所述点云特征图中的所述待卷积特征信息进行卷积处理，生成输入至第二层卷积模块的特征图；
    基于所述神经网络中第k层卷积模块对应的稀疏矩阵，确定输入至所述第k层卷积模块的特征图中的待卷积特征信息，利用所述神经网络的第k层卷积模块，对所述第k层卷积模块的特征图中的待卷积特征信息进行卷积处理，生成输入至第k+1层卷积模块的特征图，其中，k为大于1、小于n的正整数，n为所述神经网络的卷积模块的总层数；
    基于所述神经网络中第n层卷积模块对应的稀疏矩阵，确定输入至所述第n层卷积模块的特征图中的待卷积特征信息，利用所述神经网络的第n层卷积模块，对所述第n层卷积模块的特征图中的待卷积特征信息进行卷积处理，得到所述目标场景中包括的目标对象的三维检测数据。
  15. 根据权利要求12或13所述的处理方法，其特征在于，所述基于所述点云特征图和所述至少一个稀疏矩阵，利用检测目标对象的神经网络，确定所述目标场景中包括的目标对象的三维检测数据，包括：
    针对所述神经网络中除最后一层卷积模块之外的其他每一层卷积模块，基于该层卷积模块对应的稀疏矩阵和输入至该层卷积模块的特征图，确定该层卷积模块对应的卷积向量；基于该层卷积模块对应的所述卷积向量，确定输入至下一层卷积模块的特征图；
    基于最后一层卷积模块对应的稀疏矩阵和输入至最后一层卷积模块的特征图，确定最后一层卷积模块对应的卷积向量；基于最后一层卷积模块对应的所述卷积向量，确定所述目标场景中包括的目标对象的三维检测数据。
  16. 一种智能行驶控制方法，包括：
    利用设置在行驶装置上的目标雷达采集点云数据；
    基于如权利要求1至15任一项所述的点云数据的处理方法对所述点云数据进行检测处理，得到检测结果；
    基于所述检测结果，控制所述行驶装置。
  17. 一种点云数据的处理装置，包括：
    获取模块，用于获取目标雷达在第一安装姿态下采集目标场景得到的第一点云数据；
    转换模块，用于基于预先确定的第一变换矩阵，将所述第一点云数据转换为预设标准姿态下的第二点云数据；
    检测模块，用于对所述第二点云数据执行检测任务，得到目标检测结果。
  18. 一种智能行驶控制装置，包括：
    获取模块，用于获取设置在行驶装置上的目标雷达采集的点云数据；
    处理模块，用于基于如权利要求1至15任一项所述的点云数据的处理方法对所述点云数据进行检测处理，得到检测结果；
    控制模块，用于基于所述检测结果，控制所述行驶装置。
  19. 一种计算机设备，包括处理器和存储器，所述存储器存储有所述处理器可执行的机器可读指令，所述机器可读指令被所述处理器执行时，所述处理器执行如权利要求1至16任一项所述的方法。
  20. 一种计算机可读存储介质，其上存储有计算机程序，所述计算机程序被计算机设备运行时，所述计算机设备执行如权利要求1至16任意一项所述的方法。
PCT/CN2021/102706 2020-07-22 2021-06-28 点云数据的处理方法、智能行驶控制方法及装置 WO2022017131A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010713992.8A CN113970922B (zh) 2020-07-22 2020-07-22 点云数据的处理方法、智能行驶控制方法及装置
CN202010713992.8 2020-07-22

Publications (1)

Publication Number Publication Date
WO2022017131A1 true WO2022017131A1 (zh) 2022-01-27

Family

ID=79585077

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/102706 WO2022017131A1 (zh) 2020-07-22 2021-06-28 点云数据的处理方法、智能行驶控制方法及装置

Country Status (2)

Country Link
CN (1) CN113970922B (zh)
WO (1) WO2022017131A1 (zh)

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114638953A (zh) * 2022-02-22 2022-06-17 深圳元戎启行科技有限公司 点云数据的分割方法、装置及计算机可读存储介质
CN114821251A (zh) * 2022-04-28 2022-07-29 北京大学深圳研究生院 一种点云上采样网络的确定方法及确定装置
CN114820465A (zh) * 2022-04-06 2022-07-29 合众新能源汽车有限公司 点云检测模型训练方法、装置、电子设备及存储介质
CN114998331A (zh) * 2022-07-28 2022-09-02 深圳市信润富联数字科技有限公司 Led灯罩缺陷检测方法、装置、电子设备及存储介质
CN115147685A (zh) * 2022-07-20 2022-10-04 广西大学 一种甘蔗收获机运行状态的识别预测方法
CN116030134A (zh) * 2023-02-14 2023-04-28 长沙智能驾驶研究院有限公司 定位方法、装置、设备、可读存储介质及程序产品
CN117392632A (zh) * 2023-12-11 2024-01-12 中交第二公路勘察设计研究院有限公司 一种道路要素变化监测方法及装置
CN118376290A (zh) * 2024-06-21 2024-07-23 浙江省白马湖实验室有限公司 一种基于推耙机点云配准的高斯预测煤堆体积计算方法

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2008256498A (ja) * 2007-04-04 2008-10-23 Nikon Corp 計測装置およびその計測方法
WO2018101707A1 (ko) * 2016-11-30 2018-06-07 현대엠엔소프트 주식회사 지엔에스에스 시스템의 위치결정 오차 보정 장치 및 방법
CN108562913A (zh) * 2018-04-19 2018-09-21 武汉大学 一种基于三维激光雷达的无人艇假目标检测方法
CN110059579A (zh) * 2019-03-27 2019-07-26 北京三快在线科技有限公司 用于活体检验的方法和装置，电子设备和存储介质
CN110222626A (zh) * 2019-06-03 2019-09-10 宁波智能装备研究院有限公司 一种基于深度学习算法的无人驾驶场景点云目标标注方法
CN111414798A (zh) * 2019-02-03 2020-07-14 沈阳工业大学 基于rgb-d图像的头部姿态检测方法及系统

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3525131A1 (en) * 2018-02-09 2019-08-14 Bayerische Motoren Werke Aktiengesellschaft Methods and apparatuses for object detection in a scene represented by depth data of a range detection sensor and image data of a camera
CN111191583B (zh) * 2019-12-30 2023-08-25 郑州科技学院 基于卷积神经网络的空间目标识别系统及方法

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2008256498A (ja) * 2007-04-04 2008-10-23 Nikon Corp 計測装置およびその計測方法
WO2018101707A1 (ko) * 2016-11-30 2018-06-07 현대엠엔소프트 주식회사 지엔에스에스 시스템의 위치결정 오차 보정 장치 및 방법
CN108562913A (zh) * 2018-04-19 2018-09-21 武汉大学 一种基于三维激光雷达的无人艇假目标检测方法
CN111414798A (zh) * 2019-02-03 2020-07-14 沈阳工业大学 基于rgb-d图像的头部姿态检测方法及系统
CN110059579A (zh) * 2019-03-27 2019-07-26 北京三快在线科技有限公司 用于活体检验的方法和装置，电子设备和存储介质
CN110222626A (zh) * 2019-06-03 2019-09-10 宁波智能装备研究院有限公司 一种基于深度学习算法的无人驾驶场景点云目标标注方法

Cited By (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114638953A (zh) * 2022-02-22 2022-06-17 深圳元戎启行科技有限公司 点云数据的分割方法、装置及计算机可读存储介质
CN114638953B (zh) * 2022-02-22 2023-12-22 深圳元戎启行科技有限公司 点云数据的分割方法、装置及计算机可读存储介质
CN114820465A (zh) * 2022-04-06 2022-07-29 合众新能源汽车有限公司 点云检测模型训练方法、装置、电子设备及存储介质
CN114820465B (zh) * 2022-04-06 2024-04-26 合众新能源汽车股份有限公司 点云检测模型训练方法、装置、电子设备及存储介质
CN114821251A (zh) * 2022-04-28 2022-07-29 北京大学深圳研究生院 一种点云上采样网络的确定方法及确定装置
CN114821251B (zh) * 2022-04-28 2024-04-12 北京大学深圳研究生院 一种点云上采样网络的确定方法及确定装置
CN115147685A (zh) * 2022-07-20 2022-10-04 广西大学 一种甘蔗收获机运行状态的识别预测方法
CN114998331A (zh) * 2022-07-28 2022-09-02 深圳市信润富联数字科技有限公司 Led灯罩缺陷检测方法、装置、电子设备及存储介质
CN116030134A (zh) * 2023-02-14 2023-04-28 长沙智能驾驶研究院有限公司 定位方法、装置、设备、可读存储介质及程序产品
CN117392632A (zh) * 2023-12-11 2024-01-12 中交第二公路勘察设计研究院有限公司 一种道路要素变化监测方法及装置
CN117392632B (zh) * 2023-12-11 2024-03-15 中交第二公路勘察设计研究院有限公司 一种道路要素变化监测方法及装置
CN118376290A (zh) * 2024-06-21 2024-07-23 浙江省白马湖实验室有限公司 一种基于推耙机点云配准的高斯预测煤堆体积计算方法

Also Published As

Publication number Publication date
CN113970922A (zh) 2022-01-25
CN113970922B (zh) 2024-06-14

Similar Documents

Publication Publication Date Title
WO2022017131A1 (zh) 点云数据的处理方法、智能行驶控制方法及装置
US11565407B2 (en) Learning device, learning method, learning model, detection device and grasping system
JP6364049B2 (ja) 点群データに基づく車両輪郭検出方法、装置、記憶媒体およびコンピュータプログラム
CN110363058B (zh) 使用单触发卷积神经网络的用于避障的三维对象定位
US20220011122A1 (en) Trajectory prediction method and device
JP6745328B2 (ja) 点群データを復旧するための方法及び装置
US20210390329A1 (en) Image processing method, device, movable platform, unmanned aerial vehicle, and storage medium
CN111639663B (zh) 多传感器数据融合的方法
WO2020211655A1 (zh) 激光粗配准方法、装置、移动终端及存储介质
JP7439153B2 (ja) 全方位場所認識のためのリフトされたセマンティックグラフ埋め込み
WO2022228391A1 (zh) 一种终端设备定位方法及其相关设备
CN117975436A (zh) 基于多模态融合和可变形注意力的三维目标检测方法
CN116188893A (zh) 基于bev的图像检测模型训练及目标检测方法和装置
CN114120067A (zh) 一种物体识别方法、装置、设备及介质
JP2019164836A (ja) 学習装置、学習方法、学習モデル、検出装置及び把持システム
CN116129234A (zh) 一种基于注意力的4d毫米波雷达与视觉的融合方法
US20240221215A1 (en) High-precision vehicle positioning
CN111401190A (zh) 车辆检测方法、装置、计算机设备和存储介质
CN113420648A (zh) 一种具有旋转适应性的目标检测方法及系统
WO2022017129A1 (zh) 目标对象检测方法、装置、电子设备及存储介质
CN116844124A (zh) 三维目标检测框标注方法、装置、电子设备和存储介质
CN116681733A (zh) 一种空间非合作目标近距离实时位姿跟踪方法
CN114648639B (zh) 一种目标车辆的检测方法、系统及装置
CN112747752B (zh) 基于激光里程计的车辆定位方法、装置、设备和存储介质
CN114462545A (zh) 一种基于语义slam的地图构建方法及装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21847270

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 04/07/2023)

122 Ep: pct application non-entry in european phase

Ref document number: 21847270

Country of ref document: EP

Kind code of ref document: A1