WO2023193401A1 - 点云检测模型训练方法、装置、电子设备及存储介质 - Google Patents

点云检测模型训练方法、装置、电子设备及存储介质 Download PDF

Info

Publication number
WO2023193401A1
WO2023193401A1 PCT/CN2022/117359 CN2022117359W WO2023193401A1 WO 2023193401 A1 WO2023193401 A1 WO 2023193401A1 CN 2022117359 W CN2022117359 W CN 2022117359W WO 2023193401 A1 WO2023193401 A1 WO 2023193401A1
Authority
WO
WIPO (PCT)
Prior art keywords
point cloud
cloud data
detection model
sample point
module
Prior art date
Application number
PCT/CN2022/117359
Other languages
English (en)
French (fr)
Inventor
赵天坤
Original Assignee
合众新能源汽车股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 合众新能源汽车股份有限公司 filed Critical 合众新能源汽车股份有限公司
Publication of WO2023193401A1 publication Critical patent/WO2023193401A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/048—Activation functions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10028—Range image; Depth image; 3D point clouds
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20112—Image segmentation details
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T10/00—Road transport of goods or passengers
    • Y02T10/10—Internal combustion engine [ICE] based vehicles
    • Y02T10/40—Engine management systems

Definitions

  • the present application relates to the field of autonomous driving technology, and in particular to a point cloud detection model training method, device, electronic equipment and storage medium.
  • Point cloud data refers to a set of vectors in a three-dimensional coordinate system. Spatial information is recorded in the form of points, and each point contains three-dimensional coordinates. Depending on the data collection capabilities of point cloud collection equipment, some point cloud data may also contain color information (RGB) or reflection intensity information (Intensity). Taking point cloud data collected through lidar as an example, point cloud data includes the position coordinates and reflection intensity information of points in three-dimensional space. Point cloud data is widely used for target detection and recognition in the field of autonomous driving. For example, it is used for target detection and recognition in autonomous driving fields such as cars and drones. In the application process of point cloud data, point cloud detection is usually used to detect targets based on point cloud data.
  • RGB color information
  • Intensity reflection intensity information
  • Point cloud detection methods are usually 3D point cloud target detection methods based on bird's-eye view or 3D point cloud target detection methods based on pointpillar. These methods roughly divide the continuous original point cloud into anchor points. Important detailed features will be discarded when extracting features, and the location and classification prediction accuracy of the target object is low.
  • Embodiments of the present application provide a point cloud detection model training method, device, electronic equipment and storage medium to solve the problem that in related technologies, the point cloud target detection method discards important detailed features during feature extraction, resulting in the location of the target object. Problems with low classification prediction accuracy.
  • embodiments of the present application provide a point cloud detection model training method, including:
  • sample point cloud data includes: first sample point cloud data and second sample point cloud data;
  • the intermediate point cloud detection model is trained to obtain a target point cloud detection model.
  • the obtaining sample point cloud data includes:
  • Preprocess the road point cloud data remove point cloud data that does not meet the preset conditions in the road point cloud data, and obtain target road point cloud data;
  • the point cloud features of the first dimension are used as the sample point cloud data.
  • the initial point cloud detection model includes: a feature extraction module, a feature processing module and a detection module, and the first sample point cloud data corresponds to a first initial label;
  • the training of the initial point cloud detection model based on the first sample point cloud data to obtain an intermediate point cloud detection model includes:
  • the trained initial point cloud detection model is used as the intermediate point cloud detection model.
  • calling the feature extraction module to perform feature extraction on the first sample point cloud data to obtain image mapping features of the first sample point cloud data includes:
  • the point cloud features of the second dimension are mapped to the two-dimensional image to obtain the image mapping features.
  • the initial point cloud detection model further includes: a feature connection module located between the detection module and the feature processing module,
  • the detection module includes: a position detection module, a size detection module, an angle detection module and a heat map detection module.
  • the first initial label includes: the initial position, initial size, rotation angle and heat map of the object,
  • the heat map detection module is called to process the point cloud connection features to obtain a predicted heat map of the first sample point cloud data.
  • calculating the loss value of the initial point cloud detection model based on the first initial label and the first predicted label includes:
  • the second sample point cloud data includes the annotation center point and annotation category of the annotation frame, and the second sample point cloud data corresponds to a second initial label;
  • Training the intermediate point cloud detection model based on the second sample point cloud data and the auxiliary network for category prediction and center point prediction to obtain a target point cloud detection model including:
  • the trained intermediate point cloud detection model that does not include the auxiliary network is used as the Target point cloud detection model.
  • the intermediate point cloud detection model includes: a feature processing module, which is composed of a preset number of convolution modules, and the auxiliary network is connected to the convolution module,
  • the method further includes:
  • the model parameters corresponding to the feature processing module are adjusted based on the second loss value.
  • embodiments of the present application provide a point cloud detection model training device, including:
  • a sample point cloud data acquisition module is used to obtain sample point cloud data;
  • the sample point cloud data includes: first sample point cloud data and second sample point cloud data;
  • An intermediate detection model acquisition module is used to train an initial point cloud detection model based on the first sample point cloud data to obtain an intermediate point cloud detection model
  • a target detection model acquisition module is configured to train the intermediate point cloud detection model based on the second sample point cloud data and an auxiliary network for category prediction and center point prediction, to obtain a target point cloud detection model.
  • the sample point cloud data acquisition module includes:
  • a road point cloud data acquisition unit is used to obtain road point cloud data
  • a target point cloud data acquisition unit is used to preprocess the road point cloud data, remove point cloud data that does not meet the preset conditions in the road point cloud data, and obtain the target road point cloud data;
  • the target point cloud data dividing unit is used to divide the target road point cloud data into several point cloud voxels
  • a point cloud feature generation unit configured to generate a feature based on the three-dimensional coordinates of each point in the point cloud voxel, the distance between each point and the center point of the point cloud voxel, and the reflection intensity value of each point.
  • the first-dimensional point cloud feature corresponding to the point cloud voxel;
  • a sample point cloud data acquisition unit is configured to use the point cloud features of the first dimension as the sample point cloud data.
  • the initial point cloud detection model includes: a feature extraction module, a feature processing module and a detection module, and the first sample point cloud data corresponds to a first initial label;
  • the intermediate detection model acquisition module includes:
  • An image mapping feature acquisition unit configured to call the feature extraction module to process the first sample point cloud data to obtain image mapping features corresponding to the first sample point cloud data;
  • a point cloud feature acquisition unit is configured to call the feature processing module to perform feature processing on the image mapping features to obtain point cloud features of a preset size
  • a first prediction label generation unit configured to call the detection module to process the point cloud features of the preset size and generate a first prediction label of the first sample point cloud data
  • a loss value calculation unit configured to calculate the loss value of the initial point cloud detection model based on the first initial label and the first predicted label
  • An intermediate detection model acquisition unit is configured to use the trained initial point cloud detection model as the intermediate point cloud detection model when the loss value is within a preset range.
  • the image mapping feature acquisition unit includes:
  • the point cloud feature acquisition subunit is used to call the feature extraction module to process the first sample point cloud data to obtain second-dimensional point cloud features
  • Image mapping feature acquisition subunit used to map the point cloud features of the second dimension to a two-dimensional image according to the reference position of each point in the first sample point cloud data to obtain the image mapping feature .
  • the initial point cloud detection model further includes: a feature connection module located between the detection module and the feature processing module,
  • the detection module includes: a position detection module, a size detection module, an angle detection module and a heat map detection module.
  • the first initial label includes: the initial position, initial size, rotation angle and heat map of the object,
  • the point cloud connection feature acquisition subunit is used to call the feature connection module to perform feature connection processing on the point cloud features of the preset size to obtain point cloud connection features;
  • the predicted position acquisition subunit is used to call the position detection module to process the point cloud connection features to obtain the predicted position of the target object in the first sample point cloud data;
  • the predicted size acquisition subunit is used to call the size detection module to process the point cloud connection features to obtain the predicted size of the target object in the first sample point cloud data;
  • the predicted angle acquisition subunit is used to call the angle detection module to process the point cloud connection features to obtain the predicted rotation angle of the first sample point cloud data;
  • the predicted heat map acquisition subunit is used to call the heat map detection module to process the point cloud connection features to obtain the predicted heat map of the first sample point cloud data.
  • the loss value calculation unit includes:
  • a position loss value calculation subunit used to calculate a position loss value based on the initial position and the predicted position
  • a size loss value calculation subunit used to calculate a size loss value based on the initial size and the predicted size
  • Angle loss value calculation subunit used to calculate an angle loss value based on the initial rotation angle and the predicted rotation angle
  • the heat map loss value calculation subunit is used to calculate the heat map loss value based on the object heat map and the predicted heat map;
  • the model loss value acquisition subunit is used to calculate the sum of the position loss value, the size loss value, the angle loss value and the heat map loss value, and use the sum value as the initial point cloud detection The loss value of the model.
  • the second sample point cloud data includes the annotation center point and annotation category of the annotation frame, and the second sample point cloud data corresponds to a second initial label;
  • the target detection model acquisition module includes:
  • a second prediction label acquisition unit is configured to call the intermediate point cloud detection model to process the second sample point cloud data to obtain a second prediction label corresponding to the second sample point cloud data;
  • a prediction center point acquisition unit configured to call the auxiliary network to process the second sample point cloud data and obtain the prediction center point and prediction category of the prediction frame of the second sample point cloud data;
  • a first loss value calculation unit configured to calculate a first loss value of the intermediate point cloud detection model based on the second initial label and the second predicted label;
  • a second loss value calculation unit configured to calculate a second loss value of the auxiliary network based on the annotation center point, the annotation category, the prediction center point, and the prediction category;
  • a target detection model acquisition unit configured to obtain the trained model that does not include the auxiliary network when the first loss value is within a first preset range and the second loss value is within a second preset range.
  • the intermediate point cloud detection model serves as the target point cloud detection model.
  • the intermediate point cloud detection model includes: a feature processing module, which is composed of a preset number of convolution modules, and the auxiliary network is connected to the convolution module,
  • the device also includes:
  • a model parameter adjustment module configured to adjust model parameters corresponding to the feature processing module based on the second loss value when the second loss value is not within the second preset range.
  • an electronic device including:
  • a memory a processor, and a computer program stored in the memory and executable on the processor.
  • the computer program is executed by the processor, the point cloud detection model training method described in any one of the above is implemented.
  • embodiments of the present application provide a readable storage medium that, when instructions in the storage medium are executed by a processor of an electronic device, enables the electronic device to execute any of the above point cloud detection models. Training methods.
  • embodiments of the present application provide a computing processing device, including:
  • a memory having computer readable code stored therein;
  • One or more processors when the computer readable code is executed by the one or more processors, the computing processing device executes any one of the above point cloud detection model training methods.
  • embodiments of the present application provide a computer program, including computer readable code.
  • the computer readable code When the computer readable code is run on a computing processing device, it causes the computing processing device to execute the method according to any one of the above. Point cloud detection model training method.
  • sample point cloud data which includes: first sample point cloud data and second sample point cloud data
  • the initial point cloud detection model is performed based on the first sample point cloud data.
  • the intermediate point cloud detection model is obtained.
  • the intermediate point cloud detection model is trained based on the second sample point cloud data and the auxiliary network used for category prediction and center point prediction, and the target point cloud detection model is obtained.
  • the embodiment of the present application uses an auxiliary network for category prediction and center point prediction to assist in training to obtain a point cloud detection model, thereby improving the feature extraction capability of the point cloud detection model, thereby improving the prediction of target object location and classification. Accuracy.
  • Figure 1 is a step flow chart of a point cloud detection model training method provided by an embodiment of the present application
  • Figure 2 is a step flow chart of a sample point cloud data acquisition method provided by an embodiment of the present application
  • Figure 3 is a step flow chart of an intermediate point cloud detection model training method provided by an embodiment of the present application.
  • Figure 4 is a step flow chart of a target point cloud detection model training method provided by an embodiment of the present application
  • Figure 5 is a schematic structural diagram of a point cloud detection model provided by an embodiment of the present application.
  • Figure 6 is a schematic diagram of cubic interpolation provided by an embodiment of the present application.
  • Figure 7 is a schematic structural diagram of a point cloud detection model training device provided by an embodiment of the present application.
  • Figure 8 is a schematic structural diagram of an electronic device provided by an embodiment of the present application.
  • Figure 9 schematically shows a block diagram of a computing processing device for performing a method according to the invention.
  • Figure 10 schematically shows a storage unit for holding or carrying program code for implementing the method according to the invention.
  • the point cloud detection model training method may include the following steps:
  • Step 101 Obtain sample point cloud data; the sample point cloud data includes: first sample point cloud data and second sample point cloud data.
  • the embodiments of the present application can be applied to scenarios in which a point cloud detection model is trained in conjunction with an auxiliary network for center point and category prediction.
  • sample point cloud data refers to sample data used for point cloud detection model training.
  • the training process of the point cloud detection model can be divided into two stages. The first stage is the independent training stage of the point cloud detection model, and the second stage is the stage of adding an auxiliary network to assist in training the point cloud detection model.
  • the sample point cloud data is divided into the first sample point cloud data and the second sample point cloud data. Among them, “first" and "second" are only used to distinguish The data used in different model training stages has no real meaning.
  • sample point cloud data can be obtained.
  • the acquisition process of sample point cloud data can be described in detail as follows in conjunction with Figure 2.
  • the sample point cloud data acquisition method may include: step 201, step 202, and step 203. , step 204 and step 205.
  • Step 201 Obtain road point cloud data.
  • the road point cloud data can be obtained through the lidar installed on the vehicle.
  • an RSU Robot Side Unit
  • a microwave radar device to convert the information Upload to the cloud in real time to obtain road point cloud data.
  • the road point cloud data is a collection of unordered points, each point has 4 dimensions, namely (x, y, z, i), where (x, y, z) is the spatial position corresponding to each point, and i is the reflection intensity value corresponding to each point.
  • road point cloud data can be obtained based on the lidar installed on the vehicle.
  • step 202 is executed.
  • Step 202 Preprocess the road point cloud data, remove point cloud data that does not meet the preset conditions in the road point cloud data, and obtain target road point cloud data.
  • the target road point cloud data refers to the remaining point cloud data after removing the point cloud data that does not meet the preset conditions in the road point cloud data.
  • the road point cloud data can be preprocessed to remove point cloud data that does not meet the preset conditions in the road point cloud data to obtain the target road point cloud data.
  • the original point cloud collected by the acquisition device first needs to be preprocessed to obtain a point set that meets the requirements. For example, for the original point cloud, remove the nan values (null values), or remove the points with very large values to filter the point cloud noise.
  • point cloud preprocessing please refer to the prior art. In the embodiments of this application, the technical solution adopted for point cloud preprocessing is not limited and will not be described again here.
  • step 203 After preprocessing the road point cloud data to remove point cloud data that does not meet the preset conditions in the road point cloud data and obtaining the target road point cloud data, step 203 is executed.
  • Step 203 Divide the target road point cloud data into several point cloud voxels.
  • the point cloud collected by point cloud collection equipment is a point cloud in a three-dimensional irregular space area.
  • the data of the points in the area of interest in the large cube area determined previously is obtained to facilitate subsequent point cloud detection and point cloud segmentation of the point cloud in the area of interest.
  • the coordinates of points within the area of interest can be expressed by (x, y, z), where xmin ⁇ x ⁇ xmax, ymin ⁇ y ⁇ ymax, zmin ⁇ z ⁇ zmax, and the unit is meters .
  • points in the region of interest are determined based on point cloud quality. For example, if the point cloud far away from the vehicle is sparse and the number of points hitting the vehicle is small, you can set the minimum number of points to a smaller value (for example, the point value is equal to 5), and then find the corresponding number of points based on this number. , and determine a spatial area based on a maximum distance point. In some embodiments of the present application, for the same point cloud quality (such as point clouds collected by the same point cloud collection device), this distance can be predetermined by the quality of the collected point cloud data and will not change during the application process.
  • the method of determining the region of interest please refer to the method of determining the region of interest used in point cloud detection or point cloud segmentation solutions in the prior art.
  • the specific implementation method of determining the region of interest is not limited.
  • the point cloud data in the area of interest can be divided into several point cloud voxels. Specifically, the points in the area of interest can be divided along the x-axis and y-axis directions respectively. , divided into several columnar point cloud voxels, and no division is made in the z-axis direction. For example, the points in the area of interest can be divided into cuboid voxels along the x-axis and y-axis directions respectively. The z-axis direction is not divided.
  • each divided voxel can be expressed as [x v , y v ,zmax-zmin], where x v represents the length of the voxel along the x-axis direction, y v represents the length of the voxel along the y-axis direction, zmax-zmin represents the height of the voxel along the z-axis direction, the unit is meters.
  • W ⁇ H columnar voxels can be divided, where,
  • W (xmax-xmin)/x v
  • H (ymax-ymin)/y v .
  • the area of interest is divided into 512 ⁇ 250 columnar voxels.
  • step 204 After dividing the target road point cloud data into several point cloud voxels, step 204 is executed.
  • Step 204 Generate the point cloud volume based on the three-dimensional coordinates of each point in the point cloud voxel, the distance between each point and the center point of the point cloud voxel, and the reflection intensity value of each point.
  • the first-dimensional point cloud feature corresponding to the pixel.
  • each point cloud voxel contains a certain number of points (if there are point cloud voxels that do not contain points, then Discard this voxel), the distance between each point in each point cloud voxel and the center point of the point cloud voxel can be calculated, represented by xc, yc, zc.
  • the point cloud features of the first dimension of the point cloud voxel can be generated.
  • the first dimension is 7 dimensions, and the generated point cloud features are (x, y, z, i, xc, yc, zc).
  • step 205 After generating the first-dimensional point cloud features of each point cloud voxel, step 205 is performed.
  • Step 205 Use the point cloud features of the first dimension as the sample point cloud data.
  • the first-dimensional point cloud features of each point cloud voxel can be used as sample point cloud data for training the point cloud detection model.
  • step 102 After obtaining the sample point cloud data, perform step 102.
  • Step 102 Train an initial point cloud detection model based on the first sample point cloud data to obtain an intermediate point cloud detection model.
  • the initial point cloud detection model refers to the model to be trained for detecting target objects in point clouds.
  • the intermediate point cloud detection model refers to the point cloud detection model obtained after the first stage of training of the initial point cloud detection model using sample point cloud data.
  • the initial point cloud detection model can be trained based on the first sample point cloud data to obtain an intermediate point cloud detection model.
  • the specific training process can be described in detail as follows in conjunction with Figure 3.
  • the intermediate point cloud detection model training method may include: step 301, step 302, Step 303, step 304 and step 305.
  • Step 301 Call the feature extraction module to process the first sample point cloud data to obtain image mapping features corresponding to the first sample point cloud data.
  • the initial point cloud detection model may include: a feature extraction module, a feature processing module and a detection module.
  • the feature extraction module is a VFE module
  • the feature processing module consists of three Block modules and a CBR module.
  • the detection module consists of four detection modules, namely the four CBR modules located after the ConCat module in Figure 5.
  • the first sample point cloud data corresponds to a first initial label.
  • the first initial label includes: the initial position, initial size, rotation angle and heat map of the object marked in the first sample point cloud data.
  • the first sample point cloud data in the sample point cloud data can be input to the initial point cloud detection model, and then the feature extraction module is called to process the first sample point cloud data to obtain Image mapping features corresponding to the first sample point cloud data.
  • the feature extraction module can be called to process the first sample point cloud data to obtain the point cloud features of the second dimension, and then, according to the reference position of each point in the first sample point cloud data, the second dimension The point cloud features are mapped to the two-dimensional image to obtain the image mapping features.
  • the feature extraction module is constructed by serial connection of a fully connected layer, a normalization layer and a one-dimensional maximum pooling layer MaxPool1D. Finally, it outputs N ⁇ D dimensional features, where D is the dimension output by the fully connected layer. Among them, D represents the number of feature dimensions of each columnar voxel, N is the number of point cloud voxels, the input first sample point cloud data is the point cloud feature of N*K*7 dimensions, and K is the point cloud volume.
  • Cloud features are point cloud features in the second dimension.
  • the reference position refers to the original position corresponding to each point in the point cloud voxel.
  • the second-dimensional points can be processed according to the reference position of each point in the point cloud voxels.
  • the cloud features are mapped to the two-dimensional image to obtain the image mapping features corresponding to the point cloud voxels.
  • the N*D-dimensional features are mapped to the image features. Due to the sparsity of the point cloud, some locations will not There are corresponding voxels, and the features at these positions are set to 0.
  • the final feature dimensions are (W, H, D), where W and H represent the width and height of the image respectively.
  • step 302 After calling the feature extraction module to process the first sample point cloud data and obtaining the image mapping features corresponding to the first sample point cloud data, step 302 is executed.
  • Step 302 Call the feature processing module to perform feature processing on the image mapping features to obtain point cloud features of a preset size.
  • the feature processing module can be called to perform feature processing on the image mapping features to obtain point cloud features of a preset size.
  • the backbone of the point cloud detection model The network can adopt a convolutional neural network commonly used in the existing technology.
  • the backbone network further includes: three cascaded feature processing modules of different scales, wherein each feature extraction module includes: a different number of feature mapping modules (CBR ), an upsampling layer, and, a feature mapping module (CBR).
  • the number of convolutional layers in the feature mapping module (CBR) included in each feature extraction module can be 3, 5, and 5 respectively.
  • the feature mapping module can be composed of a convolutional layer, a batch normalization layer, and a Relu activation function level. Joint composition. Taking the input feature size as W ⁇ H as an example, the feature sizes output by these three feature extraction modules are (W/2,H/2), (W/4,H/4), (W/8, H/8); The feature splicing layer is used to splice the features output by the above three feature extraction modules. In this way, after inputting the image mapping features of size 1 to the backbone network, the above three feature extraction modules perform convolution operation, upsampling, normalization and activation processing on the input bird's-eye view features respectively, so that points of preset size can be obtained Cloud characteristics.
  • step 303 is executed.
  • Step 303 Call the detection module to process the point cloud features of the preset size and generate a first prediction label of the first sample point cloud data.
  • the detection module can be called to process the point cloud features of the preset size to generate a first prediction label of the first sample point cloud data.
  • the first prediction label includes: the predicted position, predicted size, predicted rotation angle and predicted heat map of the object in the predicted first sample point cloud data.
  • the initial point cloud detection model further includes: a feature connection module, the feature connection module is located between the detection module and the feature processing module, and the detection module includes: Position detection module, size detection module, angle detection module and heat map detection module.
  • the first initial label includes: the initial position, initial size, rotation angle and heat map of the object.
  • the above step 303 may include:
  • Sub-step S1 Call the feature connection module to perform feature connection processing on the point cloud features of the preset size to obtain point cloud connection features.
  • the initial point cloud detection model may also include a feature connection module, which is located between the detection module and the feature processing module.
  • the ConCat module is the feature connection module.
  • the feature connection module can be called to perform feature connection processing on the point cloud features of the preset size to obtain a point cloud connection feature.
  • the image mapping features are processed through three Block, CBR, and upsampling can output point cloud features of three preset sizes.
  • the ConCat module can splice and fuse these three preset size point cloud features to obtain a point cloud connection feature.
  • the point cloud connection features can be used as input to the detection module, and the following sub-steps S2, S3, S4 and S5 are executed respectively.
  • Sub-step S2 Call the position detection module to process the point cloud connection features to obtain the predicted position of the target object in the first sample point cloud data.
  • the position detection module can be called to process the point cloud connection features to predict the predicted position of the target object in the first sample point cloud data.
  • Sub-step S3 Call the size detection module to process the point cloud connection features to obtain the predicted size of the target object in the first sample point cloud data.
  • the size detection module can be called to process the point cloud connection features to predict the predicted size of the target object in the first sample point cloud data.
  • Sub-step S4 Call the angle detection module to process the point cloud connection features to obtain the predicted rotation angle of the first sample point cloud data.
  • the angle detection module can be called to process the point cloud connection features to predict the predicted rotation angle of the target object in the first sample point cloud data.
  • Sub-step S5 Call the heat map detection module to process the point cloud connection features to obtain a predicted heat map of the first sample point cloud data.
  • the heat map detection module can be called to process the point cloud connection features to predict the predicted heat map of the target object in the first sample point cloud data.
  • the four detection modules output the predicted heatmap (heatmap), predicted position (center), predicted size (size) and predicted rotation angle (angle) respectively.
  • the predicted heatmap, predicted position, predicted size and predicted The rotation angles together constitute the first predicted label of the first sample point cloud data.
  • step 304 is performed.
  • Step 304 Calculate the loss value of the initial point cloud detection model according to the first initial label and the first predicted label.
  • the loss value of the initial point cloud detection model can be calculated based on the first initial label and the first predicted label.
  • the loss value of the initial point cloud detection model includes: position loss value , size loss value, angle loss value and heat map loss value.
  • the position loss value can be calculated based on the initial position and predicted position.
  • the size loss value can be calculated based on the initial size and predicted size.
  • the angle loss value can be calculated based on the initial rotation angle and the predicted rotation angle.
  • the heat map loss value can be calculated based on the object heat map and predicted heat map.
  • the position prediction loss, size prediction loss, and rotation angle prediction loss can be expressed by mean square error.
  • the position prediction loss of the multi-task neural network is represented by the mean square error of the predicted values of the target object position (such as spatial position coordinates) of all the voxelized point cloud training samples and the true value of the target object position in the sample label.
  • the size prediction loss of the multi-task neural network is represented by the mean square error of the predicted value of the target size (such as three-dimensional size) of all the voxelized point cloud training samples and the true value of the target size in the sample label; by The mean square error between the predicted value of the target rotation angle of all the voxelized point cloud training samples and the true value of the target rotation angle in the sample label represents the rotation angle prediction loss of the multi-task neural network.
  • the heat map prediction loss is calculated using a pixel-by-pixel focal loss function (ie, focal loss function).
  • the position of the target object is p
  • the key points (P x , P y ) on the heat map are obtained, and the calculated data are distributed to the heat map through Gaussian kernel. If the Gaussian kernels of multiple targets overlap, the maximum value will be taken.
  • the formula of the Gaussian kernel can be expressed as:
  • x and y are the enumerated step block positions in the image to be detected, is the target scale adaptive variance, and Y xyc is the Gaussian heat map data representation of each key point after Gaussian kernel mapping.
  • step 305 is executed.
  • Step 305 If the loss value is within the preset range, use the trained initial point cloud detection model as the intermediate point cloud detection model.
  • the loss value of the initial point cloud detection model After the loss value of the initial point cloud detection model is calculated, it can be determined whether the loss value is within the preset range.
  • the trained initial point cloud detection model is used as the intermediate point cloud detection model, and the first stage of the model training task is completed.
  • step 103 After training the initial point cloud detection model based on the first sample point cloud data to obtain the intermediate point cloud detection model, step 103 is performed.
  • Step 103 Train the intermediate point cloud detection model according to the second sample point cloud data and the auxiliary network used for category prediction and center point prediction to obtain a target point cloud detection model.
  • the intermediate point cloud detection model After training the intermediate point cloud detection model, the intermediate point cloud detection model can be trained based on the second sample point cloud data and the auxiliary network used for category prediction and center point prediction until the model converges to obtain the target point cloud detection Model. Due to the addition of the auxiliary network, the feature extraction capability of the point cloud detection model can be greatly improved.
  • the target point cloud detection model training method may include: step 401, step 402, Step 403, step 404 and step 405.
  • Step 401 Call the intermediate point cloud detection model to process the second sample point cloud data to obtain a second prediction label corresponding to the second sample point cloud data.
  • the second sample point cloud data includes the annotation center point and the annotation category of the annotation frame, and the second sample point cloud data corresponds to a second initial label, and the second initial label is the same as mentioned in the above steps. is similar to the first initial tag, and this embodiment will not describe the second initial tag in detail.
  • the intermediate point cloud detection model can be called to process the second sample point cloud data to obtain the second prediction label corresponding to the second sample point cloud data.
  • the second prediction label is the same as the above-mentioned
  • the first prediction label mentioned in the steps is similar, and this embodiment will not elaborate on the second prediction label and the acquisition method of the second prediction label in detail.
  • Step 402 Call the auxiliary network to process the second sample point cloud data to obtain the prediction center point and prediction category of the prediction frame of the second sample point cloud data.
  • the auxiliary network can be called to process the second sample point cloud data to predict the prediction center point and prediction category of the prediction frame of the second sample point cloud data.
  • the prediction for categories can be based on point-by-point classification supervision: the features extracted from each block are upsampled so that their size becomes (W, H). Previously, the mapping relationship between image features and point cloud voxels was recorded. Through this relationship, the image features are mapped to the center point of the points in each point cloud voxel, and then through cubic interpolation, a set of corresponding original points is obtained. Feature,interpolation method is shown in Figure 6. After passing through three blocks respectively, the features of different receptive fields will be obtained. Finally, a classifier composed of a fully connected layer will be used to classify each point. During training, the category of the point comes from the labeled box (i.e., labeled box). When the point is within the box, then the category of the point is the category of the box. If the point does not belong to any box, then the point belongs to the background.
  • labeled box i.e., labeled box
  • center point prediction is to make the size of the detection frame output by the main network more consistent with the real frame. After obtaining the features of each point, the output is the distance from each point to the center point of the box. During training, only points within the box will calculate the distance to the center point, and the distance of points not within the box is set to 0.
  • Step 403 Calculate the first loss value of the intermediate point cloud detection model according to the second initial label and the second predicted label.
  • the first loss value of the intermediate point cloud detection model may be calculated based on the second initial label and the second predicted label.
  • the calculation method of the first loss value of the intermediate point cloud detection model is similar to the calculation method of the loss value of the initial point cloud detection model in the above steps.
  • the specific calculation process can refer to the calculation of the loss value of the above initial point cloud detection model. The process will not be described again in this embodiment.
  • Step 404 Calculate the second loss value of the auxiliary network based on the annotation center point, the annotation category, the prediction center point and the prediction category.
  • the second loss value of the auxiliary network can be calculated based on the label center point, label category, prediction center point, and prediction category. Specifically, the mean square error algorithm can be used to calculate the center point loss value and the category loss value, and then the two loss values are added and summed to obtain the second loss value.
  • Step 405 When the first loss value is within the first preset range, and the second loss value is within the second preset range, use the trained intermediate point cloud detection model that does not include the auxiliary network. As the target point cloud detection model.
  • the first loss value and the second loss value are obtained through the above steps, it can be determined whether the first loss value is within the first preset range, and whether the second loss value is within the second preset range.
  • the trained intermediate points that do not include the auxiliary network can be
  • the cloud detection model is used as the target point cloud detection model, that is, after the intermediate point cloud detection model converges, the auxiliary network is removed and the main network is used as the target point cloud detection model.
  • the intermediate point cloud detection model includes: a feature processing module.
  • the feature processing module is composed of a preset number of convolution modules (three shown in Figure 5, namely Block3, Block5, and Block5).
  • the auxiliary network and The convolution module is connected.
  • the model parameters corresponding to the feature processing module can be optimized and adjusted in combination with the second loss value, and training continues until the model convergence.
  • the point cloud detection model provided in this embodiment adopts the heat map prediction method, abandoning the anchor point-based prediction method, and the predicted object angle will be more accurate.
  • the point cloud detection model training method obtains sample point cloud data.
  • the sample point cloud data includes: first sample point cloud data and second sample point cloud data. Based on the first sample point cloud data pair The initial point cloud detection model is trained to obtain the intermediate point cloud detection model.
  • the intermediate point cloud detection model is trained based on the second sample point cloud data and the auxiliary network used for category prediction and center point prediction to obtain the target point cloud detection model. .
  • the embodiment of the present application uses an auxiliary network for category prediction and center point prediction to assist in training to obtain a point cloud detection model, thereby improving the feature extraction capability of the point cloud detection model, thereby improving the prediction of target object location and classification. Accuracy.
  • the point cloud detection model training device 700 may include:
  • the sample point cloud data acquisition module 710 is used to obtain sample point cloud data; the sample point cloud data includes: first sample point cloud data and second sample point cloud data;
  • the intermediate detection model acquisition module 720 is used to train the initial point cloud detection model based on the first sample point cloud data to obtain an intermediate point cloud detection model
  • the target detection model acquisition module 730 is configured to train the intermediate point cloud detection model according to the second sample point cloud data and the auxiliary network used for category prediction and center point prediction to obtain a target point cloud detection model.
  • sample point cloud data acquisition module 710 includes:
  • a road point cloud data acquisition unit is used to obtain road point cloud data
  • a target point cloud data acquisition unit is used to preprocess the road point cloud data, remove point cloud data that does not meet the preset conditions in the road point cloud data, and obtain the target road point cloud data;
  • the target point cloud data dividing unit is used to divide the target road point cloud data into several point cloud voxels
  • a point cloud feature generation unit configured to generate a feature based on the three-dimensional coordinates of each point in the point cloud voxel, the distance between each point and the center point of the point cloud voxel, and the reflection intensity value of each point.
  • the first-dimensional point cloud feature corresponding to the point cloud voxel;
  • a sample point cloud data acquisition unit is configured to use the point cloud features of the first dimension as the sample point cloud data.
  • the initial point cloud detection model includes: a feature extraction module, a feature processing module and a detection module, and the first sample point cloud data corresponds to a first initial label;
  • the intermediate detection model acquisition module 720 includes:
  • An image mapping feature acquisition unit configured to call the feature extraction module to process the first sample point cloud data to obtain image mapping features corresponding to the first sample point cloud data;
  • a point cloud feature acquisition unit is configured to call the feature processing module to perform feature processing on the image mapping features to obtain point cloud features of a preset size
  • a first prediction label generation unit configured to call the detection module to process the point cloud features of the preset size and generate a first prediction label of the first sample point cloud data
  • a loss value calculation unit configured to calculate the loss value of the initial point cloud detection model based on the first initial label and the first predicted label
  • An intermediate detection model acquisition unit is configured to use the trained initial point cloud detection model as the intermediate point cloud detection model when the loss value is within a preset range.
  • the image mapping feature acquisition unit includes:
  • the point cloud feature acquisition subunit is used to call the feature extraction module to process the first sample point cloud data to obtain second-dimensional point cloud features
  • Image mapping feature acquisition subunit used to map the point cloud features of the second dimension to a two-dimensional image according to the reference position of each point in the first sample point cloud data to obtain the image mapping feature .
  • the initial point cloud detection model also includes: a feature connection module, the feature connection module is located between the detection module and the feature processing module,
  • the detection module includes: a position detection module, a size detection module, an angle detection module and a heat map detection module.
  • the first initial label includes: the initial position, initial size, rotation angle and heat map of the object,
  • the point cloud connection feature acquisition subunit is used to call the feature connection module to perform feature connection processing on the point cloud features of the preset size to obtain point cloud connection features;
  • the predicted position acquisition subunit is used to call the position detection module to process the point cloud connection features to obtain the predicted position of the target object in the first sample point cloud data;
  • the predicted size acquisition subunit is used to call the size detection module to process the point cloud connection features to obtain the predicted size of the target object in the first sample point cloud data;
  • the predicted angle acquisition subunit is used to call the angle detection module to process the point cloud connection features to obtain the predicted rotation angle of the first sample point cloud data;
  • the loss value calculation unit includes:
  • a position loss value calculation subunit used to calculate a position loss value based on the initial position and the predicted position
  • a size loss value calculation subunit used to calculate a size loss value based on the initial size and the predicted size
  • Angle loss value calculation subunit used to calculate an angle loss value based on the initial rotation angle and the predicted rotation angle
  • the heat map loss value calculation subunit is used to calculate the heat map loss value based on the object heat map and the predicted heat map;
  • the model loss value acquisition subunit is used to calculate the sum of the position loss value, the size loss value, the angle loss value and the heat map loss value, and use the sum value as the initial point cloud detection The loss value of the model.
  • the second sample point cloud data includes the annotation center point and annotation category of the annotation frame, and the second sample point cloud data corresponds to a second initial label;
  • the target detection model acquisition module 730 includes:
  • a second prediction label acquisition unit is configured to call the intermediate point cloud detection model to process the second sample point cloud data to obtain a second prediction label corresponding to the second sample point cloud data;
  • a prediction center point acquisition unit configured to call the auxiliary network to process the second sample point cloud data and obtain the prediction center point and prediction category of the prediction frame of the second sample point cloud data;
  • a first loss value calculation unit configured to calculate a first loss value of the intermediate point cloud detection model based on the second initial label and the second predicted label;
  • a second loss value calculation unit configured to calculate a second loss value of the auxiliary network based on the annotation center point, the annotation category, the prediction center point, and the prediction category;
  • a target detection model acquisition unit configured to obtain the trained model that does not include the auxiliary network when the first loss value is within a first preset range and the second loss value is within a second preset range.
  • the intermediate point cloud detection model serves as the target point cloud detection model.
  • the intermediate point cloud detection model includes: a feature processing module, which is composed of a preset number of convolution modules, and the auxiliary network is connected to the convolution module,
  • the device also includes:
  • a model parameter adjustment module configured to adjust model parameters corresponding to the feature processing module based on the second loss value when the second loss value is not within the second preset range.
  • the point cloud detection model training device obtains sample point cloud data.
  • the sample point cloud data includes: first sample point cloud data and second sample point cloud data. Based on the first sample point cloud data pair The initial point cloud detection model is trained to obtain the intermediate point cloud detection model.
  • the intermediate point cloud detection model is trained based on the second sample point cloud data and the auxiliary network used for category prediction and center point prediction to obtain the target point cloud detection model. .
  • the embodiment of the present application uses an auxiliary network for category prediction and center point prediction to assist in training to obtain a point cloud detection model, thereby improving the feature extraction capability of the point cloud detection model, thereby improving the prediction of target object location and classification. Accuracy.
  • An embodiment of the present application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor.
  • the computer program is executed by the processor, the above-mentioned Point cloud detection model training method.
  • FIG. 8 shows a schematic structural diagram of an electronic device 800 according to an embodiment of the present invention.
  • the electronic device 800 includes a central processing unit (CPU) 801, which can be loaded into a random access memory (RAM) 803 according to computer program instructions stored in a read-only memory (ROM) 802 or from a storage unit 808.
  • CPU central processing unit
  • RAM random access memory
  • ROM read-only memory
  • FIG. 8 shows a schematic structural diagram of an electronic device 800 according to an embodiment of the present invention.
  • the electronic device 800 includes a central processing unit (CPU) 801, which can be loaded into a random access memory (RAM) 803 according to computer program instructions stored in a read-only memory (ROM) 802 or from a storage unit 808.
  • RAM 803 random access memory
  • various programs and data required for the operation of the electronic device 800 can also be stored.
  • CPU 801, ROM 802 and RAM 803 are connected to each other via bus 804.
  • An input/output (I/O) interface 805 is also connected
  • the I/O interface 805 includes: an input unit 806, such as a keyboard, mouse, microphone, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a disk. , optical disk, etc.; and communication unit 809, such as network card, modem, wireless communication transceiver, etc.
  • the communication unit 809 allows the electronic device 800 to exchange information/data with other devices through a computer network such as the Internet and/or various telecommunications networks.
  • the various processes and processing described above may be executed by the processing unit 801.
  • the method of any of the above embodiments can be implemented as a computer software program, which is tangibly included in a computer-readable medium, such as the storage unit 808.
  • part or all of the computer program may be loaded and/or installed onto the electronic device 800 via the ROM 802 and/or the communication unit 809 .
  • a computer program is loaded into RAM 803 and executed by CPU 801, one or more actions in the methods described above may be performed.
  • Embodiments of the present application provide a computer-readable storage medium.
  • a computer program is stored on the computer-readable storage medium.
  • the computer program is executed by a processor, each process of the above point cloud detection model training method embodiment is implemented, and the same can be achieved. The technical effects will not be repeated here to avoid repetition.
  • the computer-readable storage medium is such as read-only memory (ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk, etc.
  • the device embodiments described above are only illustrative.
  • the units described as separate components may or may not be physically separated.
  • the components shown as units may or may not be physical units, that is, they may be located in One location, or it can be distributed across multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Persons of ordinary skill in the art can understand and implement the method without any creative effort.
  • Various component embodiments of the present invention may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof.
  • a microprocessor or a digital signal processor (DSP) may be used in practice to implement some or all functions of some or all components in a computing processing device according to embodiments of the present invention.
  • the invention may also be implemented as an apparatus or apparatus program (eg, computer program and computer program product) for performing part or all of the methods described herein.
  • Such a program implementing the present invention may be stored on a computer-readable medium, or may be in the form of one or more signals. Such signals may be downloaded from an Internet website, or provided on a carrier signal, or in any other form.
  • Figure 9 shows a computing processing device in which a method according to the invention can be implemented.
  • the computing processing device conventionally includes a processor 910 and a computer program product or computer readable medium in the form of memory 920.
  • Memory 920 may be electronic memory such as flash memory, EEPROM (Electrically Erasable Programmable Read Only Memory), EPROM, hard disk, or ROM.
  • the memory 920 has a storage space 930 for program code 931 for executing any method steps in the above-described methods.
  • the storage space 930 for program codes may include individual program codes 931 respectively used to implement various steps in the above method. These program codes can be read from or written into one or more computer program products.
  • These computer program products include program code carriers such as hard disks, compact disks (CDs), memory cards or floppy disks. Such computer program products are typically portable or fixed storage units as described with reference to FIG. 10 .
  • the storage unit may have storage segments, storage spaces, etc. arranged similarly to the memory 920 in the computing processing device of FIG. 9 .
  • the program code may, for example, be compressed in a suitable form.
  • the storage unit includes computer readable code 931', ie code that can be read by, for example, a processor such as 910, which code, when executed by a computing processing device, causes the computing processing device to perform the methods described above. various steps.
  • any reference signs placed between parentheses shall not be construed as limiting the claim.
  • the word “comprising” does not exclude the presence of elements or steps not listed in a claim.
  • the word “a” or “an” preceding an element does not exclude the presence of a plurality of such elements.
  • the invention may be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the element claim enumerating several means, several of these means may be embodied by the same item of hardware.
  • the use of the words first, second, third, etc. does not indicate any order. These words can be interpreted as names.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Computational Linguistics (AREA)
  • Biomedical Technology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Biophysics (AREA)
  • Software Systems (AREA)
  • Mathematical Physics (AREA)
  • Quality & Reliability (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Image Analysis (AREA)

Abstract

一种点云检测模型训练方法、装置、电子设备及存储介质。所述方法包括：获取样本点云数据；所述样本点云数据包括：第一样本点云数据和第二样本点云数据（101）；基于所述第一样本点云数据对初始点云检测模型进行训练，得到中间点云检测模型（102）；根据所述第二样本点云数据和用于进行类别预测和中心点预测的辅助网络，对所述中间点云检测模型进行训练，得到目标点云检测模型（103）；以提高点云检测模型的特征提取能力，从而提高目标物在位置和分类方面的预测准确度。

Description

点云检测模型训练方法、装置、电子设备及存储介质
本申请要求在2022年04月06日提交中国专利局、申请号为202210357107.6、发明名称为“点云检测模型训练方法、装置、电子设备及存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及自动驾驶技术领域，尤其涉及一种点云检测模型训练方法、装置、电子设备及存储介质。
背景技术
点云数据(point cloud data)是指在一个三维坐标系统中的一组向量的集合。空间信息以点的形式记录，每一个点包含有三维坐标。根据点云采集设备的数据采集能力的差异，有些点云数据中可能还含有颜色信息(RGB)或反射强度信息(Intensity)等。以通过激光雷达采集的点云数据为例，点云数据包括三维空间中点的位置坐标和反射强度信息。点云数据广泛用于自动驾驶领域中进行目标物检测和识别。例如，用于汽车、无人机等自动驾驶领域的目标物检测和识别。在点云数据的应用过程中，通常要采用点云检测，以基于点云数据进行目标物检测。
现有的点云检测方法，通常是基于鸟瞰图的3D点云目标检测方法或者基于pointpillar的3D点云目标检测方法，这些方法都是将连续的原始点云粗略的划分成了锚点，在提取特征时会丢弃重要的细节特征，目标物的位置和分类预测准确度较低。
发明内容
本申请实施例提供一种点云检测模型训练方法、装置、电子设备及存储介质，以解决相关技术中点云目标检测方法在进行特征提取时会丢弃重要的细节特征，导致目标物的位置和分类预测准确度较低的问题。
为了解决上述技术问题，本申请实施例是这样实现的：
第一方面，本申请实施例提供了一种点云检测模型训练方法，包括：
获取样本点云数据；所述样本点云数据包括：第一样本点云数据和第二样本点云数据；
基于所述第一样本点云数据对初始点云检测模型进行训练，得到中间点云检测模型；
根据所述第二样本点云数据和用于进行类别预测和中心点预测的辅助网络，对所述中间点云检测模型进行训练，得到目标点云检测模型。
可选地，所述获取样本点云数据，包括：
获取道路点云数据；
对所述道路点云数据进行预处理，去除所述道路点云数据中不符合预设条件的点云数据，得到目标道路点云数据；
将所述目标道路点云数据划分为若干个点云体素；
根据所述点云体素中的每个点的三维坐标、每个点与所述点云体素的中心点的距离、及每个点的反射强度值，生成所述点云体素对应的第一维度的点云特征；
将所述第一维度的点云特征作为所述样本点云数据。
可选地，所述初始点云检测模型包括：特征提取模块、特征处理模块和检测模块，所述第一样本点云数据对应于一个第一初始标签；
所述基于所述第一样本点云数据对初始点云检测模型进行训练，得到中间点云检测模型，包括：
调用所述特征提取模块对所述第一样本点云数据进行处理，得到所述第一样本点云数据对应的图像映射特征；
调用所述特征处理模块对所述图像映射特征进行特征处理，得到预设尺寸的点云特征；
调用所述检测模块对所述预设尺寸的点云特征进行处理，生成所述第一样本点云数据的第一预测标签；
根据所述第一初始标签和所述第一预测标签，计算得到所述初始点云检测模型的损失值；
在所述损失值处于预设范围内的情况下，将训练后的初始点云检测模型 作为所述中间点云检测模型。
可选地，所述调用所述特征提取模块对所述第一样本点云数据进行特征提取，得到所述第一样本点云数据的图像映射特征，包括：
调用所述特征提取模块对所述第一样本点云数据进行处理，得到第二维度的点云特征；
根据所述第一样本点云数据中每个点的基准位置，将所述第二维度的点云特征映射至二维图像上，得到所述图像映射特征。
可选地，所述初始点云检测模型还包括：特征连接模块，所述特征连接模块位于所述检测模块和所述特征处理模块之间，
所述检测模块包括：位置检测模块、尺寸检测模块、角度检测模块和热力图检测模块，所述第一初始标签包括：物体的初始位置、初始尺寸、旋转角度和物体热力图，
所述调用所述检测模块对所述预设尺寸的点云特征进行处理，生成所述第一样本点云数据的第一预测标签，包括：
调用所述特征连接模块对所述预设尺寸的点云特征进行特征连接处理，得到点云连接特征；
调用所述位置检测模块对所述点云连接特征进行处理，得到所述第一样本点云数据中目标物的预测位置；
调用所述尺寸检测模块对所述点云连接特征进行处理，得到所述第一样本点云数据中目标物的预测尺寸；
调用所述角度检测模块对所述点云连接特征进行处理，得到所述第一样本点云数据的预测旋转角度；
调用所述热力图检测模块对所述点云连接特征进行处理，得到所述第一样本点云数据的预测热力图。
可选地，所述根据所述第一初始标签和所述第一预测标签，计算得到所述初始点云检测模型的损失值，包括：
根据所述初始位置和所述预测位置，计算得到位置损失值；
根据所述初始尺寸和所述预测尺寸，计算得到尺寸损失值；
根据所述初始旋转角度和所述预测旋转角度，计算得到角度损失值；
根据所述物体热力图和所述预测热力图，计算得到热力图损失值；
计算所述位置损失值、所述尺寸损失值、所述角度损失值和所述热力图损失值的和值，并将该和值作为所述初始点云检测模型的损失值。
可选地，所述第二样本点云数据包含有标注框的标注中心点和标注类别，且所述第二样本点云数据对应于一个第二初始标签；
所述根据所述第二样本点云数据和用于进行类别预测和中心点预测的辅助网络，对所述中间点云检测模型进行训练，得到目标点云检测模型，包括：
调用所述中间点云检测模型对所述第二样本点云数据进行处理，得到所述第二样本点云数据对应的第二预测标签；
调用所述辅助网络对所述第二样本点云数据进行处理，得到所述第二样本点云数据的预测框的预测中心点和预测类别；
根据所述第二初始标签和所述第二预测标签，计算得到所述中间点云检测模型的第一损失值；
根据所述标注中心点、所述标注类别、所述预测中心点和所述预测类别，计算得到所述辅助网络的第二损失值；
在所述第一损失值处于第一预设范围内，且所述第二损失值处于第二预设范围内的情况下，将训练后的不包含辅助网络的中间点云检测模型作为所述目标点云检测模型。
可选地，所述中间点云检测模型包括：特征处理模块，所述特征处理模块由预设个数的卷积模块组成，所述辅助网络与所述卷积模块连接，
在所述根据所述标注中心点、所述标注类别、所述预测中心点和所述预测类别，计算得到所述辅助网络的第二损失值之后，还包括：
在所述第二损失值不处于所述第二预设范围内的情况下，基于所述第二损失值调整所述特征处理模块对应的模型参数。
第二方面，本申请实施例提供了一种点云检测模型训练装置，包括：
样本点云数据获取模块，用于获取样本点云数据；所述样本点云数据包括：第一样本点云数据和第二样本点云数据；
中间检测模型获取模块，用于基于所述第一样本点云数据对初始点云检 测模型进行训练，得到中间点云检测模型；
目标检测模型获取模块，用于根据所述第二样本点云数据和用于进行类别预测和中心点预测的辅助网络，对所述中间点云检测模型进行训练，得到目标点云检测模型。
可选地，所述样本点云数据获取模块包括：
道路点云数据获取单元，用于获取道路点云数据；
目标点云数据获取单元，用于对所述道路点云数据进行预处理，去除所述道路点云数据中不符合预设条件的点云数据，得到目标道路点云数据；
目标点云数据划分单元，用于将所述目标道路点云数据划分为若干个点云体素；
点云特征生成单元，用于根据所述点云体素中的每个点的三维坐标、每个点与所述点云体素的中心点的距离、及每个点的反射强度值，生成所述点云体素对应的第一维度的点云特征；
样本点云数据获取单元，用于将所述第一维度的点云特征作为所述样本点云数据。
可选地，所述初始点云检测模型包括：特征提取模块、特征处理模块和检测模块，所述第一样本点云数据对应于一个第一初始标签；
所述中间检测模型获取模块包括：
图像映射特征获取单元，用于调用所述特征提取模块对所述第一样本点云数据进行处理，得到所述第一样本点云数据对应的图像映射特征；
点云特征获取单元，用于调用所述特征处理模块对所述图像映射特征进行特征处理，得到预设尺寸的点云特征；
第一预测标签生成单元，用于调用所述检测模块对所述预设尺寸的点云特征进行处理，生成所述第一样本点云数据的第一预测标签；
损失值计算单元，用于根据所述第一初始标签和所述第一预测标签，计算得到所述初始点云检测模型的损失值；
中间检测模型获取单元，用于在所述损失值处于预设范围内的情况下，将训练后的初始点云检测模型作为所述中间点云检测模型。
可选地，所述图像映射特征获取单元包括：
点云特征获取子单元，用于调用所述特征提取模块对所述第一样本点云数据进行处理，得到第二维度的点云特征；
图像映射特征获取子单元，用于根据所述第一样本点云数据中每个点的基准位置，将所述第二维度的点云特征映射至二维图像上，得到所述图像映射特征。
可选地，所述初始点云检测模型还包括：特征连接模块，所述特征连接模块位于所述检测模块和所述特征处理模块之间，
所述检测模块包括：位置检测模块、尺寸检测模块、角度检测模块和热力图检测模块，所述第一初始标签包括：物体的初始位置、初始尺寸、旋转角度和物体热力图，
所述第一预测标签生成单元包括：
点云连接特征获取子单元，用于调用所述特征连接模块对所述预设尺寸的点云特征进行特征连接处理，得到点云连接特征；
预测位置获取子单元，用于调用所述位置检测模块对所述点云连接特征进行处理，得到所述第一样本点云数据中目标物的预测位置；
预测尺寸获取子单元，用于调用所述尺寸检测模块对所述点云连接特征进行处理，得到所述第一样本点云数据中目标物的预测尺寸；
预测角度获取子单元，用于调用所述角度检测模块对所述点云连接特征进行处理，得到所述第一样本点云数据的预测旋转角度；
预测热力图获取子单元，用于调用所述热力图检测模块对所述点云连接特征进行处理，得到所述第一样本点云数据的预测热力图。
可选地，所述损失值计算单元包括：
位置损失值计算子单元，用于根据所述初始位置和所述预测位置，计算得到位置损失值；
尺寸损失值计算子单元，用于根据所述初始尺寸和所述预测尺寸，计算得到尺寸损失值；
角度损失值计算子单元，用于根据所述初始旋转角度和所述预测旋转角度，计算得到角度损失值；
热力图损失值计算子单元，用于根据所述物体热力图和所述预测热力 图，计算得到热力图损失值；
模型损失值获取子单元，用于计算所述位置损失值、所述尺寸损失值、所述角度损失值和所述热力图损失值的和值，并将该和值作为所述初始点云检测模型的损失值。
可选地，所述第二样本点云数据包含有标注框的标注中心点和标注类别，且所述第二样本点云数据对应于一个第二初始标签；
所述目标检测模型获取模块包括：
第二预测标签获取单元，用于调用所述中间点云检测模型对所述第二样本点云数据进行处理，得到所述第二样本点云数据对应的第二预测标签；
预测中心点获取单元，用于调用所述辅助网络对所述第二样本点云数据进行处理，得到所述第二样本点云数据的预测框的预测中心点和预测类别；
第一损失值计算单元，用于根据所述第二初始标签和所述第二预测标签，计算得到所述中间点云检测模型的第一损失值；
第二损失值计算单元，用于根据所述标注中心点、所述标注类别、所述预测中心点和所述预测类别，计算得到所述辅助网络的第二损失值；
目标检测模型获取单元，用于在所述第一损失值处于第一预设范围内，且所述第二损失值处于第二预设范围内的情况下，将训练后的不包含辅助网络的中间点云检测模型作为所述目标点云检测模型。
可选地，所述中间点云检测模型包括：特征处理模块，所述特征处理模块由预设个数的卷积模块组成，所述辅助网络与所述卷积模块连接，
所述装置还包括：
模型参数调整模块，用于在所述第二损失值不处于所述第二预设范围内的情况下，基于所述第二损失值调整所述特征处理模块对应的模型参数。
第三方面，本申请实施例提供了一种电子设备，包括：
存储器、处理器及存储在所述存储器上并可在所述处理器上运行的计算机程序，所述计算机程序被所述处理器执行时实现上述任一项所述的点云检测模型训练方法。
第四方面，本申请实施例提供了一种可读存储介质，当所述存储介质中的指令由电子设备的处理器执行时，使得电子设备能够执行上述任一项所述 的点云检测模型训练方法。
第五方面，本申请实施例提供了一种计算处理设备，包括：
存储器，其中存储有计算机可读代码；
一个或多个处理器，当所述计算机可读代码被所述一个或多个处理器执行时，所述计算处理设备执行上述任一项所述的点云检测模型训练方法。
第六方面，本申请实施例提供了一种计算机程序，包括计算机可读代码，当所述计算机可读代码在计算处理设备上运行时，导致所述计算处理设备执行根据上述任一项所述的点云检测模型训练方法。
在本申请实施例中，通过获取样本点云数据，样本点云数据包括：第一样本点云数据和第二样本点云数据，基于第一样本点云数据对初始点云检测模型进行训练，得到中间点云检测模型，根据第二样本点云数据和用于进行类别预测和中心点预测的辅助网络对中间点云检测模型进行训练，得到目标点云检测模型。本申请实施例通过用于进行类别预测和中心点预测的辅助网络，辅助训练得到点云检测模型，从而可以提高点云检测模型的特征提取能力，从而可以提高目标物在位置和分类方面的预测准确度。
上述说明仅是本发明技术方案的概述，为了能够更清楚了解本发明的技术手段，而可依照说明书的内容予以实施，并且为了让本发明的上述和其它目的、特征和优点能够更明显易懂，以下特举本发明的具体实施方式。
附图说明
为了更清楚地说明本发明实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作一简单地介绍，显而易见地，下面描述中的附图是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1为本申请实施例提供的一种点云检测模型训练方法的步骤流程图；
图2为本申请实施例提供的一种样本点云数据获取方法的步骤流程图；
图3为本申请实施例提供的一种中间点云检测模型训练方法的步骤流程图；
图4为本申请实施例提供的一种目标点云检测模型训练方法的步骤流 程图；
图5为本申请实施例提供的一种点云检测模型的结构示意图；
图6为本申请实施例提供的一种三次插值的示意图；
图7为本申请实施例提供的一种点云检测模型训练装置的结构示意图；
图8为本申请实施例提供的一种电子设备的结构示意图；
图9示意性地示出了用于执行根据本发明的方法的计算处理设备的框图；以及
图10示意性地示出了用于保持或者携带实现根据本发明的方法的程序代码的存储单元。
具体实施例
为使本发明实施例的目的、技术方案和优点更加清楚，下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。
参照图1，示出了本申请实施例提供的一种点云检测模型训练方法的步骤流程图，如图1所示，该点云检测模型训练方法可以包括如下步骤：
步骤101：获取样本点云数据；所述样本点云数据包括：第一样本点云数据和第二样本点云数据。
本申请实施例可以应用于结合用于进行中心点和类别预测的辅助网络训练点云检测模型的场景中。
在本实施例中，样本点云数据是指用于进行点云检测模型训练的样本数据。在本示例中，点云检测模型的训练过程可以分为两个阶段，第一个阶段为点云检测模型的单独训练阶段，第二个阶段为加入辅助网络辅助训练点云检测模型的阶段。为了区分两个阶段的样本点云数据，所以将样本点云数据划分为第一样本点云数据和第二样本点云数据，其中，“第一”和“第二”仅是用于区分不同模型训练阶段采用的数据，并无实质含义。
在对点云检测模型进行训练时，可以获取样本点云数据。对于样本点云 数据的获取流程可以结合图2进行如下详细描述。
参照图2，示出了本申请实施例提供的一种样本点云数据获取方法的步骤流程图，如图2所示，该样本点云数据获取方法可以包括：步骤201、步骤202、步骤203、步骤204和步骤205。
步骤201：获取道路点云数据。
在本实施例中，道路点云数据可以是通过车辆上安装的激光雷达获取的，在实际应用中，还可以采用RSU(Road Side Unit，路侧单元)整合高清摄像头和微波雷达装置，将信息实时上传给云端，以获取道路点云数据。
在本示例中，道路点云数据为由一组无序点组成的集合，每个点的维度为4维，分别为(x,y,z,i)，其中，(x,y,z)为每个点对应的空间位置，i为每个点对应的反射强度值。
在自动驾驶场景中，可以根据车辆上安装的激光雷达获取道路点云数据。
在获取到道路点云数据之后，执行步骤202。
步骤202：对所述道路点云数据进行预处理，去除所述道路点云数据中不符合预设条件的点云数据，得到目标道路点云数据。
目标道路点云数据是指去除了道路点云数据中不符合预设条件的点云数据之后，剩余的点云数据。
在获取到道路点云数据之后，可以对道路点云数据进行预处理，以去除道路点云数据中不符合预设条件的点云数据，以得到目标道路点云数据，具体地，对于点云采集设备采集的原始点云，首先需要进行点云预处理，以获得符合要求的点集。例如，对于原始点云，去除其中nan值(空值)，或者，去除其中数值非常大的点，以过滤点云噪声。点云预处理的具体实施方案可以参见现有技术，本申请实施例中对点云预处理采用的技术方案不做限定，此处亦不再赘述。
在对道路点云数据进行预处理去除道路点云数据中不符合预设条件的点云数据，得到目标道路点云数据之后，执行步骤203。
步骤203：将所述目标道路点云数据划分为若干个点云体素。
点云采集设备(如激光雷达传感器)采集到的点云是三维的不规则空间 区域内的点，在生成样本点云数据之前，首先需要从中确定一个规则的空间区域内的点云。例如，通过限定x,y和z方向的坐标范围，取一块大的立方体区域中的点云，其余的舍弃，这个立方体区域的大小可以表示为：[xmax-xmin，ymax-ymin，zmax-zmin]，其中，xmax和xmin分别表示x方向的坐标最大值和最小值，ymax和ymin分别表示y方向的坐标最大值和最小值，zmax和zmin分别表示z方向的坐标最大值和最小值。
进一步的，获取前文确定的大立方体区域中感兴趣区域内的点的数据，便于后续对感兴趣区域内的点云进行点云检测和点云分割。本申请的一些实施例中，感兴趣区域内的点的坐标可以通过(x,y,z)表示，其中，xmin<x<xmax,ymin<y<ymax，zmin<z<zmax，单位是米。
本申请的一些实施例中，感兴趣区域中的点根据点云质量确定。例如，距离车辆较远位置的点云比较稀疏，打到车上的点数较少，可以设置最小点数为一个较小数值(例如：点数值等于5)，然后，根据这个点数找到相应数量的点，并根据一个最大距离的点，确定一个空间区域。本申请的一些实施例中，对于同样的点云质量(如同样的点云采集设备采集的点云)，这个距离可以通过采集点云数据的质量预先确定，在应用过程中不再改变。
感兴趣区域的确定方法可以参见现有技术中点云检测或点云分割方案中采用的确定感兴趣区域的方法，本申请实施例中，对确定感兴趣区域的具体实施方式不做限定。
在获取到感兴趣区域内的点之后，可以将感兴趣区域内的点云数据划分为若干个点云体素，具体地，可以将感兴趣区域内的点分别沿着x轴和y轴方向，划分成若干个柱状的点云体素，z轴方向不做划分。例如，可以将感兴趣区域内的点分别沿着x轴和y轴方向，划分成长方体体素，z轴方向不做划分，划分得到的每个体素的大小可以表示为[x v,y v,zmax-zmin]，其中，x v表示体素沿x轴方向的长度，y v表示体素沿y轴方向的长度，zmax-zmin表示体素沿z轴方向的高度，单位是米。按照前述柱状体素生成方法，对应一个感兴趣区域，将可以划分得到W×H个柱状体素，其中，
W＝(xmax-xmin)/x v，H＝(ymax-ymin)/y v。
以感兴趣区域中x的范围为(0,102.4)，y的范围为(0,50)，z的范围为 (0,100)，柱状体素大小为0.2×0.2×100为例,则x轴方向柱状体素个数w等于(102.4-0)/0.2＝512，y轴方向柱状体素个数H等于(50-0)/0.2＝250，则感兴趣区域被划分为512×250个柱状体素。
可以理解地，上述示例仅是为了更好地理解本申请实施例的技术方案而列举的示例，不作为对本实施例的唯一限制。
在将目标道路点云数据划分为若干个点云体素之后，执行步骤204。
步骤204：根据所述点云体素中的每个点的三维坐标、每个点与所述点云体素的中心点的距离、及每个点的反射强度值，生成所述点云体素对应的第一维度的点云特征。
在将目标道路点云数据划分为若干个点云体素之后，可以根据点云体素中每个点的三维坐标，每个点与点云体素的中心点的距离、及每个点对应的反射强度值，生成每个点云体素的第一维度的点云特征，具体地，在每个点云体素中包含一定数量的点(若存在不包含点的点云体素，则舍弃此体素)，可以计算每个点云体素中每个点与点云体素的中心点之间的距离，用xc，yc，zc表示。然后根据每个点的三维坐标(x，y，z)、计算得到的距离(xc，yc，zc)和反射强度值i，即可生成点云体素的第一维度的点云特征，该第一维度即为7维度，生成的点云特征为(x，y，z，i，xc，yc，zc)。
在生成每个点云体素的第一维度的点云特征之后，执行步骤205。
步骤205：将所述第一维度的点云特征作为所述样本点云数据。
在生成每个点云体素的第一维度的点云特征之后，则可以将第一维度的点云特征作为对点云检测模型进行训练的样本点云数据。
在得到样本点云数据之后，执行步骤102。
步骤102：基于所述第一样本点云数据对初始点云检测模型进行训练，得到中间点云检测模型。
初始点云检测模型是指待训练的用于进行点云中目标物检测的模型。
中间点云检测模型是指采用样本点云数据对初始点云检测模型进行第一阶段的训练之后，得到的点云检测模型。
在获取到样本点云数据之后，可以基于第一样本点云数据对初始点云检测模型进行训练，以得到中间点云检测模型。具体地训练过程，可以结合图 3进行如下详细描述。
参照图3，示出了本申请实施例提供的一种中间点云检测模型训练方法的步骤流程图，如图3所示，该中间点云检测模型训练方法可以包括：步骤301、步骤302、步骤303、步骤304和步骤305。
步骤301：调用所述特征提取模块对所述第一样本点云数据进行处理，得到所述第一样本点云数据对应的图像映射特征。
在本实施例中，初始点云检测模型可以包括：特征提取模块、特征处理模块和检测模块，如图5所示，特征提取模块为VFE模块，特征处理模块由三个Block模块、CBR模块组成，检测模块由四个检测模块组成，即图5中位于ConCat模块后的四个CBR模块。
第一样本点云数据对应于一个第一初始标签，该第一初始标签包括：第一样本点云数据中标注的物体的初始位置、初始尺寸、旋转角度和物体热力图。
在获取到样本点云数据之后，可以将样本点云数据中的第一样本点云数据输入至初始点云检测模型，然后调用特征提取模块对第一样本点云数据进行处理，以得到第一样本点云数据对应的图像映射特征。具体地，可以调用特征提取模块对第一样本点云数据进行处理，得到第二维度的点云特征，然后，按照第一样本点云数据中每个点的基准位置，将第二维度的点云特征映射至二维图像上，以得到图像映射特征。
在本示例中，特征提取模块由全连接层、归一化层和一维最大池化层MaxPool1D串行连接构建，最后，输出N×D维度的特征，其中D为全连接层输出的维度。其中，D表示每个柱状体素的特征维度数,N为点云体素的个数，输入的第一样本点云数据为N*K*7维度的点云特征，K为点云体素中点的个数，然后将N*K*7维度的点云特征经过特征提取模块的全连接层、归一化层和一维最大池化层MaxPool1D，从而可以得到N*D维的点云特征，即第二维度的点云特征。
基准位置是指点云体素中每个点所对应的原始位置。
在调用特征处理模块对第一维度的点云特征进行处理得到点云体素的第二维度的点云特征之后，可以根据点云体素中每个点的基准位置，将第二 维度的点云特征映射至二维图像上，以得到点云体素对应的图像映射特征，具体地，将N*D维的特征映射到图像特征上，由于点云的稀疏性，某些位置将不会有体素相对应，这些位置的特征设置为0，最后形成的特征维度是(W,H,D)，其中，W和H分别表示图像的宽和高。
在调用特征提取模块对第一样本点云数据进行处理，得到第一样本点云数据对应的图像映射特征之后，执行步骤302。
步骤302：调用所述特征处理模块对所述图像映射特征进行特征处理，得到预设尺寸的点云特征。
在得到第一样本点云数据对应的图像映射特征之后，可以调用特征处理模块对图像映射特征进行特征处理，以得到预设尺寸的点云特征，具体地，其中，点云检测模型的主干网络可以采用现有技术中通用的卷积神经网络。例如，本申请的一些实施例中，如图5所示，主干网络进一步包括：三个不同尺度的级联的特征处理模块，其中，每个特征提取模块包括：不同数量的特征映射模块(CBR)，一个上采样层，以及，一个特征映射模块(CBR)。每个特征提取模块包括的特征映射模块(CBR)中的卷积层数量可以分别是3、5、5，特征映射模块(CBR)可以由卷积层、批量归一化层和Relu激活函数级联构成。以输入特征的大小为W×H为例，这三个特征提取模块输出的特征的尺寸分别是(W/2,H/2)，(W/4,H/4)，(W/8,H/8)；所述特征拼接层用于将上述三个特征提取模块输出的特征进行拼接。这样，将大小为图像映射特征输入至主干网络之后，上述三个特征提取模块分别对输入的鸟瞰图特征进行卷积运算、上采样、归一化和激活处理，从而可以得到预设尺寸的点云特征。
在得到预设尺寸的点云特征之后，执行步骤303。
步骤303：调用所述检测模块对所述预设尺寸的点云特征进行处理，生成所述第一样本点云数据的第一预测标签。
在得到预设尺寸的点云特征之后，可以调用检测模块对预设尺寸的点云特征进行处理，以生成第一样本点云数据的第一预测标签。该第一预测标签包括：预测的第一样本点云数据中的物体的预测位置、预测尺寸、预测旋转角度和预测热力图。对于生成第一预测标签的过程可以结合下述具体实现方 式进行详细描述。
在本申请的一种具体实现方式中，所述初始点云检测模型还包括：特征连接模块，所述特征连接模块位于所述检测模块和所述特征处理模块之间，所述检测模块包括：位置检测模块、尺寸检测模块、角度检测模块和热力图检测模块，所述第一初始标签包括：物体的初始位置、初始尺寸、旋转角度和物体热力图，上述步骤303可以包括：
子步骤S1：调用所述特征连接模块对所述预设尺寸的点云特征进行特征连接处理，得到点云连接特征。
在本实施例中，初始点云检测模型还可以包括特征连接模块，该特征连接模块位于检测模块和特征处理模块之间，如图5所示，ConCat模块即为特征连接模块。
在得到预设尺寸的点云特征之后，可以调用特征连接模块对预设尺寸的点云特征进行特征连接处理，以得到一个点云连接特征，如图5所示，图像映射特征分别经过三个Block、CBR、上采样可以输出三个预设尺寸的点云特征，ConCat模块可以对这三个预设尺寸的点云特征进行拼接融合，以得到一个点云连接特征。
在得到点云连接特征之后，可以将点云连接特征作为检测模块的输入，分别执行下述子步骤S2、子步骤S3、子步骤S4和子步骤S5。
子步骤S2：调用所述位置检测模块对所述点云连接特征进行处理，得到所述第一样本点云数据中目标物的预测位置。
在得到点云连接特征之后，可以调用位置检测模块对点云连接特征进行处理，以预测第一样本点云数据中目标物的预测位置。
子步骤S3：调用所述尺寸检测模块对所述点云连接特征进行处理，得到所述第一样本点云数据中目标物的预测尺寸。
在得到点云连接特征之后，可以调用尺寸检测模块对点云连接特征进行处理，以预测第一样本点云数据中目标物的预测尺寸。
子步骤S4：调用所述角度检测模块对所述点云连接特征进行处理，得到所述第一样本点云数据的预测旋转角度。
在得到点云连接特征之后，可以调用角度检测模块对点云连接特征进行 处理，以预测第一样本点云数据中目标物的预测旋转角度。
子步骤S5：调用所述热力图检测模块对所述点云连接特征进行处理，得到所述第一样本点云数据的预测热力图。
在得到点云连接特征之后，可以调用热力图检测模块对点云连接特征进行处理，以预测第一样本点云数据中目标物的预测热力图。
如图5所示，通过四个检测模块分别输出预测热力图(heatmap)、预测位置(center)、预测尺寸(size)和预测旋转角度(angle)，预测热力图、预测位置、预测尺寸和预测旋转角度共同构成了第一样本点云数据的第一预测标签。
在得到第一预测标签之后，执行步骤304。
步骤304：根据所述第一初始标签和所述第一预测标签，计算得到所述初始点云检测模型的损失值。
在得到第一预测标签之后，可以根据第一初始标签和第一预测标签，计算得到初始点云检测模型的损失值，在本示例中，初始点云检测模型的损失值即包括：位置损失值、尺寸损失值、角度损失值和热力图损失值，具体地，可以根据初始位置和预测位置，计算得到位置损失值。可以根据初始尺寸和预测尺寸，计算得到尺寸损失值。可以根据初始旋转角度和预测旋转角度，计算得到角度损失值。可以根据物体热力图和预测热力图，计算得到热力图损失值。然后将这四个损失值求和得到初始点云检测模型的损失值。
本申请的一些实施例中，位置预测损失、大小预测损失和旋转角度预测损失可以采用均方误差表达。例如，通过所有所述体素化点云训练样本的目标物位置(如空间位置坐标)的预测值和样本标签中的目标物位置真实值的均方误差，表示多任务神经网络的位置预测损失；通过所有所述体素化点云训练样本的目标物大小(如立体尺寸)的预测值和样本标签中的目标物大小真实值的均方误差，表示多任务神经网络的大小预测损失；通过所有所述体素化点云训练样本的目标物旋转角度的预测值和样本标签中的目标物旋转角度真实值的均方误差，表示多任务神经网络的旋转角度预测损失。
本申请的一些实施例中，所述热力图预测损失采用逐像素的focal loss损失函数(即焦点损失函数)计算。
假设目标物的位置为p,经过下采样计算后得到热力图上的关键点(P x,P y)，通过高斯核将计算出的数据分布到热力图上。如果多个目标物的高斯核重叠，那么将取最大值，高斯核的公式可以表示为：
Figure PCTCN2022117359-appb-000001
其中，x和y为待检测图像中枚举的步长块位置，
Figure PCTCN2022117359-appb-000002
为目标尺度自适应方差，Y xyc为高斯核映射之后的每个关键点的高斯热图数据表示。
然后，采用逐像素的focal loss损失函数计算热力图的损失，公式如下：
Figure PCTCN2022117359-appb-000003
其中，M表示目标物总数；
Figure PCTCN2022117359-appb-000004
表示网络预测出的有目标物的可能性，取值范围为(0，1)；y xyc表示是否有目标物的真实值，取值范围为(0，1)；α和β为超参数，取值根据经验设定，例如，可以取α＝2，β＝4。
在计算得到初始点云检测模型的损失值之后，执行步骤305。
步骤305：在所述损失值处于预设范围内的情况下，将训练后的初始点云检测模型作为所述中间点云检测模型。
在计算得到初始点云检测模型的损失值之后，可以判断该损失值是否处于预设范围内。
若该损失值处于预设范围内，则将训练后的初始点云检测模型作为中间点云检测模型，至此即完成了第一阶段的模型训练任务。
在基于第一样本点云数据对初始点云检测模型进行训练得到中间点云检测模型之后，执行步骤103。
步骤103：根据所述第二样本点云数据和用于进行类别预测和中心点预测的辅助网络，对所述中间点云检测模型进行训练，得到目标点云检测模型。
在训练得到中间点云检测模型之后，可以根据第二样本点云数据和用于 进行类别预测和中心点预测的辅助网络对中间点云检测模型进行训练，直至模型收敛，以得到目标点云检测模型。由于辅助网络的加入，可以极大地提高点云检测模型的特征提取能力。
对于第二阶段的模型训练过程可以参照图4进行如下详细描述。
参照图4，示出了本申请实施例提供的一种目标点云检测模型训练方法的步骤流程图，如图4所示，该目标点云检测模型训练方法可以包括：步骤401、步骤402、步骤403、步骤404和步骤405。
步骤401：调用所述中间点云检测模型对所述第二样本点云数据进行处理，得到所述第二样本点云数据对应的第二预测标签。
在本实施例中，第二样本点云数据包含有标注框的标注中心点和标注类别，且第二样本点云数据对应于一个第二初始标签，该第二初始标签与上述步骤中提及的第一初始标签相似，本实施例对于该第二初始标签不再加以详细赘述。
在进行第二阶段的模型训练过程中，可以调用中间点云检测模型对第二样本点云数据进行处理，以得到第二样本点云数据对应的第二预测标签，该第二预测标签与上述步骤中提及的第一预测标签相似，本实施例对于该第二预测标签及第二预测标签的获取方式不再加以详细赘述。
步骤402：调用所述辅助网络对所述第二样本点云数据进行处理，得到所述第二样本点云数据的预测框的预测中心点和预测类别。
在进行第二阶段的训练过程中，可以调用辅助网络对第二样本点云数据进行处理，以预测得到第二样本点云数据的预测框的预测中心点和预测类别。
具体地，针对类别的预测可以采用逐点分类监督的方式：将每个block提取的特征上采样，使其大小变为(W,H)。此前记录了图像特征与点云体素的映射关系，通过这个关系将图像特征映射到每个点云体素中的点的中心点上，然后通过三次插值，获取每个原始点对应的一组特征，插值方法如图6所示。分别经过三个block之后，将会得到不同感受野的特征，最后经过由全连接层组成的分类器，为每个点进行分类。在训练的时候，点的类别来自标注的box(即标注框)，当点在box内，那么这个点的类别就是box的类 别，如果点不属于任何一个box，那么这个点属于背景。
中心点预测目的在于使主网络输出的检测框的大小与真实框更贴合。在获取到每个点的特征之后，输出的是每个点到box中心点的距离。在训练时，只有在box内的点才会计算到中心点的距离，不在box内的点的距离设置为0。
步骤403：根据所述第二初始标签和所述第二预测标签，计算得到所述中间点云检测模型的第一损失值。
在得到第二预测标签之后，可以根据第二初始标签和第二预测标签计算得到中间点云检测模型的第一损失值。
可以理解地，中间点云检测模型的第一损失值的计算方式与上述步骤中初始点云检测模型的损失值的计算方式相似，具体计算流程可以参照上述初始点云检测模型的损失值的计算流程，本实施例在此不再加以赘述。
步骤404：根据所述标注中心点、所述标注类别、所述预测中心点和所述预测类别，计算得到所述辅助网络的第二损失值。
在得到预测中心点和预测类别之后，可以根据标注中心点、标注类别、预测中心点和预测类别计算得到辅助网络的第二损失值。具体地，可以采用均方误差算法计算得到中心点损失值和类别损失值，然后将这两个损失值相加求和得到第二损失值。
步骤405：在所述第一损失值处于第一预设范围内，且所述第二损失值处于第二预设范围内的情况下，将训练后的不包含辅助网络的中间点云检测模型作为所述目标点云检测模型。
在通过上述步骤得到第一损失值和第二损失值之后，可以判断第一损失值是否处于第一预设范围内，并判断第二损失值是否处于第二预设范围内。
若第一损失值处于第一预设范围内，且第二损失值处于第二预设范围内，则表示中间点云检测模型收敛，此时，可以将训练后的不包含辅助网络的中间点云检测模型作为目标点云检测模型，即在中间点云检测模型收敛之后，则去掉辅助网络，将主网络作为目标点云检测模型。
如图5所示，中间点云检测模型包括：特征处理模块，特征处理模块由预设个数的卷积模块(图5所示为3个，即Block3、Block5、Block5)组成， 辅助网络与卷积模块连接，在计算得到的第二损失值未处于第二预设范围内的情况下，可以结合第二损失值对特征处理模块对应的模型参数进行优化调整，并继续进行训练，直至模型收敛。
在上述点云检测模型的训练过程中，通过加入用于进行中心点和类别预测的辅助网络，辅助优化模型参数，可以提高点云检测模型的特征提取能力，可以使预测的类别会更加准确并且位置和大小也会更贴合真正的物体。同时，在训练得到的目标点云检测模型进行预测时，辅助网络会去掉，因此不会增加主网络的耗时，因此这种辅助网络的方法非常实用。
进一步地，本实施例提供的点云检测模型采用热力图的预测方式，舍弃了基于锚点的预测方式，预测的物体角度会更加精确。
本申请实施例提供的点云检测模型训练方法，通过获取样本点云数据，样本点云数据包括：第一样本点云数据和第二样本点云数据，基于第一样本点云数据对初始点云检测模型进行训练，得到中间点云检测模型，根据第二样本点云数据和用于进行类别预测和中心点预测的辅助网络对中间点云检测模型进行训练，得到目标点云检测模型。本申请实施例通过用于进行类别预测和中心点预测的辅助网络，辅助训练得到点云检测模型，从而可以提高点云检测模型的特征提取能力，从而可以提高目标物在位置和分类方面的预测准确度。
参照图7，示出了本申请实施例提供的一种点云检测模型训练装置的结构示意图，如图7所示，该点云检测模型训练装置700可以包括：
样本点云数据获取模块710，用于获取样本点云数据；所述样本点云数据包括：第一样本点云数据和第二样本点云数据；
中间检测模型获取模块720，用于基于所述第一样本点云数据对初始点云检测模型进行训练，得到中间点云检测模型；
目标检测模型获取模块730，用于根据所述第二样本点云数据和用于进行类别预测和中心点预测的辅助网络，对所述中间点云检测模型进行训练，得到目标点云检测模型。
可选地，所述样本点云数据获取模块710包括：
道路点云数据获取单元，用于获取道路点云数据；
目标点云数据获取单元，用于对所述道路点云数据进行预处理，去除所述道路点云数据中不符合预设条件的点云数据，得到目标道路点云数据；
目标点云数据划分单元，用于将所述目标道路点云数据划分为若干个点云体素；
点云特征生成单元，用于根据所述点云体素中的每个点的三维坐标、每个点与所述点云体素的中心点的距离、及每个点的反射强度值，生成所述点云体素对应的第一维度的点云特征；
样本点云数据获取单元，用于将所述第一维度的点云特征作为所述样本点云数据。
可选地，所述初始点云检测模型包括：特征提取模块、特征处理模块和检测模块，所述第一样本点云数据对应于一个第一初始标签；
所述中间检测模型获取模块720包括：
图像映射特征获取单元，用于调用所述特征提取模块对所述第一样本点云数据进行处理，得到所述第一样本点云数据对应的图像映射特征；
点云特征获取单元，用于调用所述特征处理模块对所述图像映射特征进行特征处理，得到预设尺寸的点云特征；
第一预测标签生成单元，用于调用所述检测模块对所述预设尺寸的点云特征进行处理，生成所述第一样本点云数据的第一预测标签；
损失值计算单元，用于根据所述第一初始标签和所述第一预测标签，计算得到所述初始点云检测模型的损失值；
中间检测模型获取单元，用于在所述损失值处于预设范围内的情况下，将训练后的初始点云检测模型作为所述中间点云检测模型。
可选地，所述图像映射特征获取单元包括：
点云特征获取子单元，用于调用所述特征提取模块对所述第一样本点云数据进行处理，得到第二维度的点云特征；
图像映射特征获取子单元，用于根据所述第一样本点云数据中每个点的基准位置，将所述第二维度的点云特征映射至二维图像上，得到所述图像映射特征。
可选地，所述初始点云检测模型还包括：特征连接模块，所述特征连接 模块位于所述检测模块和所述特征处理模块之间，
所述检测模块包括：位置检测模块、尺寸检测模块、角度检测模块和热力图检测模块，所述第一初始标签包括：物体的初始位置、初始尺寸、旋转角度和物体热力图，
所述第一预测标签生成单元包括：
点云连接特征获取子单元，用于调用所述特征连接模块对所述预设尺寸的点云特征进行特征连接处理，得到点云连接特征；
预测位置获取子单元，用于调用所述位置检测模块对所述点云连接特征进行处理，得到所述第一样本点云数据中目标物的预测位置；
预测尺寸获取子单元，用于调用所述尺寸检测模块对所述点云连接特征进行处理，得到所述第一样本点云数据中目标物的预测尺寸；
预测角度获取子单元，用于调用所述角度检测模块对所述点云连接特征进行处理，得到所述第一样本点云数据的预测旋转角度；
预测热力图获取子单元，用于调用所述热力图检测模块对所述点云连接特征进行处理，得到所述第一样本点云数据的预测热力图。
可选地，所述损失值计算单元包括：
位置损失值计算子单元，用于根据所述初始位置和所述预测位置，计算得到位置损失值；
尺寸损失值计算子单元，用于根据所述初始尺寸和所述预测尺寸，计算得到尺寸损失值；
角度损失值计算子单元，用于根据所述初始旋转角度和所述预测旋转角度，计算得到角度损失值；
热力图损失值计算子单元，用于根据所述物体热力图和所述预测热力图，计算得到热力图损失值；
模型损失值获取子单元，用于计算所述位置损失值、所述尺寸损失值、所述角度损失值和所述热力图损失值的和值，并将该和值作为所述初始点云检测模型的损失值。
可选地，所述第二样本点云数据包含有标注框的标注中心点和标注类别，且所述第二样本点云数据对应于一个第二初始标签；
所述目标检测模型获取模块730包括：
第二预测标签获取单元，用于调用所述中间点云检测模型对所述第二样本点云数据进行处理，得到所述第二样本点云数据对应的第二预测标签；
预测中心点获取单元，用于调用所述辅助网络对所述第二样本点云数据进行处理，得到所述第二样本点云数据的预测框的预测中心点和预测类别；
第一损失值计算单元，用于根据所述第二初始标签和所述第二预测标签，计算得到所述中间点云检测模型的第一损失值；
第二损失值计算单元，用于根据所述标注中心点、所述标注类别、所述预测中心点和所述预测类别，计算得到所述辅助网络的第二损失值；
目标检测模型获取单元，用于在所述第一损失值处于第一预设范围内，且所述第二损失值处于第二预设范围内的情况下，将训练后的不包含辅助网络的中间点云检测模型作为所述目标点云检测模型。
可选地，所述中间点云检测模型包括：特征处理模块，所述特征处理模块由预设个数的卷积模块组成，所述辅助网络与所述卷积模块连接，
所述装置还包括：
模型参数调整模块，用于在所述第二损失值不处于所述第二预设范围内的情况下，基于所述第二损失值调整所述特征处理模块对应的模型参数。
本申请实施例提供的点云检测模型训练装置，通过获取样本点云数据，样本点云数据包括：第一样本点云数据和第二样本点云数据，基于第一样本点云数据对初始点云检测模型进行训练，得到中间点云检测模型，根据第二样本点云数据和用于进行类别预测和中心点预测的辅助网络对中间点云检测模型进行训练，得到目标点云检测模型。本申请实施例通过用于进行类别预测和中心点预测的辅助网络，辅助训练得到点云检测模型，从而可以提高点云检测模型的特征提取能力，从而可以提高目标物在位置和分类方面的预测准确度。
本申请实施例提供了一种电子设备，包括：存储器、处理器及存储在所述存储器上并可在所述处理器上运行的计算机程序，所述计算机程序被所述处理器执行时实现上述点云检测模型训练方法。
图8示出了本发明实施例的一种电子设备800的结构示意图。如图8所 示，电子设备800包括中央处理单元(CPU)801，其可以根据存储在只读存储器(ROM)802中的计算机程序指令或者从存储单元808加载到随机访问存储器(RAM)803中的计算机程序指令，来执行各种适当的动作和处理。在RAM 803中，还可存储电子设备800操作所需的各种程序和数据。CPU 801、ROM 802以及RAM 803通过总线804彼此相连。输入/输出(I/O)接口805也连接至总线804。
电子设备800中的多个部件连接至I/O接口805，包括：输入单元806，例如键盘、鼠标、麦克风等；输出单元807，例如各种类型的显示器、扬声器等；存储单元808，例如磁盘、光盘等；以及通信单元809，例如网卡、调制解调器、无线通信收发机等。通信单元809允许电子设备800通过诸如因特网的计算机网络和/或各种电信网络与其他设备交换信息/数据。
上文所描述的各个过程和处理，可由处理单元801执行。例如，上述任一实施例的方法可被实现为计算机软件程序，其被有形地包含于计算机可读介质，例如存储单元808。在一些实施例中，计算机程序的部分或者全部可以经由ROM802和/或通信单元809而被载入和/或安装到电子设备800上。当计算机程序被加载到RAM803并由CPU801执行时，可以执行上文描述的方法中的一个或多个动作。
本申请实施例提供了一种计算机可读存储介质，计算机可读存储介质上存储有计算机程序，计算机程序被处理器执行时实现上述点云检测模型训练方法实施例的各个过程，且能达到相同的技术效果，为避免重复，这里不再赘述。其中，所述的计算机可读存储介质，如只读存储器(Read-Only Memory，简称ROM)、随机存取存储器(Random Access Memory，简称RAM)、磁碟或者光盘等。
以上所描述的装置实施例仅仅是示意性的，其中所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部模块来实现本实施例方案的目的。本领域普通技术人员在不付出创造性的劳动的情况下，即可以理解并实施。
本发明的各个部件实施例可以以硬件实现，或者以在一个或者多个处理器上运行的软件模块实现，或者以它们的组合实现。本领域的技术人员应当理解，可以在实践中使用微处理器或者数字信号处理器(DSP)来实现根据本发明实施例的计算处理设备中的一些或者全部部件的一些或者全部功能。本发明还可以实现为用于执行这里所描述的方法的一部分或者全部的设备或者装置程序(例如，计算机程序和计算机程序产品)。这样的实现本发明的程序可以存储在计算机可读介质上，或者可以具有一个或者多个信号的形式。这样的信号可以从因特网网站上下载得到，或者在载体信号上提供，或者以任何其他形式提供。
例如，图9示出了可以实现根据本发明的方法的计算处理设备。该计算处理设备传统上包括处理器910和以存储器920形式的计算机程序产品或者计算机可读介质。存储器920可以是诸如闪存、EEPROM(电可擦除可编程只读存储器)、EPROM、硬盘或者ROM之类的电子存储器。存储器920具有用于执行上述方法中的任何方法步骤的程序代码931的存储空间930。例如，用于程序代码的存储空间930可以包括分别用于实现上面的方法中的各种步骤的各个程序代码931。这些程序代码可以从一个或者多个计算机程序产品中读出或者写入到这一个或者多个计算机程序产品中。这些计算机程序产品包括诸如硬盘，紧致盘(CD)、存储卡或者软盘之类的程序代码载体。这样的计算机程序产品通常为如参考图10所述的便携式或者固定存储单元。该存储单元可以具有与图9的计算处理设备中的存储器920类似布置的存储段、存储空间等。程序代码可以例如以适当形式进行压缩。通常，存储单元包括计算机可读代码931’，即可以由例如诸如910之类的处理器读取的代码，这些代码当由计算处理设备运行时，导致该计算处理设备执行上面所描述的方法中的各个步骤。
本文中所称的“一个实施例”、“实施例”或者“一个或者多个实施例”意味着，结合实施例描述的特定特征、结构或者特性包括在本发明的至少一个实施例中。此外，请注意，这里“在一个实施例中”的词语例子不一定全指同一个实施例。
在此处所提供的说明书中，说明了大量具体细节。然而，能够理解， 本发明的实施例可以在没有这些具体细节的情况下被实践。在一些实例中，并未详细示出公知的方法、结构和技术，以便不模糊对本说明书的理解。
在权利要求中，不应将位于括号之间的任何参考符号构造成对权利要求的限制。单词“包含”不排除存在未列在权利要求中的元件或步骤。位于元件之前的单词“一”或“一个”不排除存在多个这样的元件。本发明可以借助于包括有若干不同元件的硬件以及借助于适当编程的计算机来实现。在列举了若干装置的单元权利要求中，这些装置中的若干个可以是通过同一个硬件项来具体体现。单词第一、第二、以及第三等的使用不表示任何顺序。可将这些单词解释为名称。
最后应说明的是：以上实施例仅用以说明本发明的技术方案，而非对其限制；尽管参照前述实施例对本发明进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本发明各实施例技术方案的精神和范围。

Claims (13)

  1. 一种点云检测模型训练方法，包括：
    获取样本点云数据；所述样本点云数据包括：第一样本点云数据和第二样本点云数据；
    基于所述第一样本点云数据对初始点云检测模型进行训练，得到中间点云检测模型；
    根据所述第二样本点云数据和用于进行类别预测和中心点预测的辅助网络，对所述中间点云检测模型进行训练，得到目标点云检测模型。
  2. 根据权利要求1所述的方法，其中，所述获取样本点云数据，包括：
    获取道路点云数据；
    对所述道路点云数据进行预处理，去除所述道路点云数据中不符合预设条件的点云数据，得到目标道路点云数据；
    将所述目标道路点云数据划分为若干个点云体素；
    根据所述点云体素中的每个点的三维坐标、每个点与所述点云体素的中心点的距离、及每个点的反射强度值，生成所述点云体素对应的第一维度的点云特征；
    将所述第一维度的点云特征作为所述样本点云数据。
  3. 根据权利要求1所述的方法，其中，所述初始点云检测模型包括：特征提取模块、特征处理模块和检测模块，所述第一样本点云数据对应于一个第一初始标签；
    所述基于所述第一样本点云数据对初始点云检测模型进行训练，得到中间点云检测模型，包括：
    调用所述特征提取模块对所述第一样本点云数据进行处理，得到所述第一样本点云数据对应的图像映射特征；
    调用所述特征处理模块对所述图像映射特征进行特征处理，得到预设尺寸的点云特征；
    调用所述检测模块对所述预设尺寸的点云特征进行处理，生成所述第一样本点云数据的第一预测标签；
    根据所述第一初始标签和所述第一预测标签，计算得到所述初始点云检 测模型的损失值；
    在所述损失值处于预设范围内的情况下，将训练后的初始点云检测模型作为所述中间点云检测模型。
  4. 根据权利要求3所述的方法，其中，所述调用所述特征提取模块对所述第一样本点云数据进行特征提取，得到所述第一样本点云数据的图像映射特征，包括：
    调用所述特征提取模块对所述第一样本点云数据进行处理，得到第二维度的点云特征；
    根据所述第一样本点云数据中每个点的基准位置，将所述第二维度的点云特征映射至二维图像上，得到所述图像映射特征。
  5. 根据权利要求3所述的方法，其中，所述初始点云检测模型还包括：特征连接模块，所述特征连接模块位于所述检测模块和所述特征处理模块之间，
    所述检测模块包括：位置检测模块、尺寸检测模块、角度检测模块和热力图检测模块，所述第一初始标签包括：物体的初始位置、初始尺寸、旋转角度和物体热力图，
    所述调用所述检测模块对所述预设尺寸的点云特征进行处理，生成所述第一样本点云数据的第一预测标签，包括：
    调用所述特征连接模块对所述预设尺寸的点云特征进行特征连接处理，得到点云连接特征；
    调用所述位置检测模块对所述点云连接特征进行处理，得到所述第一样本点云数据中目标物的预测位置；
    调用所述尺寸检测模块对所述点云连接特征进行处理，得到所述第一样本点云数据中目标物的预测尺寸；
    调用所述角度检测模块对所述点云连接特征进行处理，得到所述第一样本点云数据的预测旋转角度；
    调用所述热力图检测模块对所述点云连接特征进行处理，得到所述第一样本点云数据的预测热力图。
  6. 根据权利要求5所述的方法，其中，所述根据所述第一初始标签和 所述第一预测标签，计算得到所述初始点云检测模型的损失值，包括：
    根据所述初始位置和所述预测位置，计算得到位置损失值；
    根据所述初始尺寸和所述预测尺寸，计算得到尺寸损失值；
    根据所述初始旋转角度和所述预测旋转角度，计算得到角度损失值；
    根据所述物体热力图和所述预测热力图，计算得到热力图损失值；
    计算所述位置损失值、所述尺寸损失值、所述角度损失值和所述热力图损失值的和值，并将该和值作为所述初始点云检测模型的损失值。
  7. 根据权利要求1所述的方法，其中，所述第二样本点云数据包含有标注框的标注中心点和标注类别，且所述第二样本点云数据对应于一个第二初始标签；
    所述根据所述第二样本点云数据和用于进行类别预测和中心点预测的辅助网络，对所述中间点云检测模型进行训练，得到目标点云检测模型，包括：
    调用所述中间点云检测模型对所述第二样本点云数据进行处理，得到所述第二样本点云数据对应的第二预测标签；
    调用所述辅助网络对所述第二样本点云数据进行处理，得到所述第二样本点云数据的预测框的预测中心点和预测类别；
    根据所述第二初始标签和所述第二预测标签，计算得到所述中间点云检测模型的第一损失值；
    根据所述标注中心点、所述标注类别、所述预测中心点和所述预测类别，计算得到所述辅助网络的第二损失值；
    在所述第一损失值处于第一预设范围内，且所述第二损失值处于第二预设范围内的情况下，将训练后的不包含辅助网络的中间点云检测模型作为所述目标点云检测模型。
  8. 根据权利要求7所述的方法，其中，所述中间点云检测模型包括：特征处理模块，所述特征处理模块由预设个数的卷积模块组成，所述辅助网络与所述卷积模块连接，
    在所述根据所述标注中心点、所述标注类别、所述预测中心点和所述预测类别，计算得到所述辅助网络的第二损失值之后，还包括：
    在所述第二损失值不处于所述第二预设范围内的情况下，基于所述第二损失值调整所述特征处理模块对应的模型参数。
  9. 一种点云检测模型训练装置，包括：
    样本点云数据获取模块，用于获取样本点云数据；所述样本点云数据包括：第一样本点云数据和第二样本点云数据；
    中间检测模型获取模块，用于基于所述第一样本点云数据对初始点云检测模型进行训练，得到中间点云检测模型；
    目标检测模型获取模块，用于根据所述第二样本点云数据和用于进行类别预测和中心点预测的辅助网络，对所述中间点云检测模型进行训练，得到目标点云检测模型。
  10. 一种电子设备，包括：
    存储器、处理器及存储在所述存储器上并可在所述处理器上运行的计算机程序，所述计算机程序被所述处理器执行时实现如权利要求1至8中任一项所述的点云检测模型训练方法。
  11. 一种可读存储介质，当所述存储介质中的指令由电子设备的处理器执行时，使得电子设备能够执行权利要求1至8任一项所述的点云检测模型训练方法。
  12. 一种计算处理设备，包括：
    存储器，其中存储有计算机可读代码；
    一个或多个处理器，当所述计算机可读代码被所述一个或多个处理器执行时，所述计算处理设备执行如权利要求1至8中任一项所述的点云检测模型训练方法。
  13. 一种计算机程序，包括计算机可读代码，当所述计算机可读代码在计算处理设备上运行时，导致所述计算处理设备执行根据权利要求1至8中任一项所述的点云检测模型训练方法。
PCT/CN2022/117359 2022-04-06 2022-09-06 点云检测模型训练方法、装置、电子设备及存储介质 WO2023193401A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202210357107.6 2022-04-06
CN202210357107.6A CN114820465B (zh) 2022-04-06 2022-04-06 点云检测模型训练方法、装置、电子设备及存储介质

Publications (1)

Publication Number Publication Date
WO2023193401A1 true WO2023193401A1 (zh) 2023-10-12

Family

ID=82534768

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/117359 WO2023193401A1 (zh) 2022-04-06 2022-09-06 点云检测模型训练方法、装置、电子设备及存储介质

Country Status (2)

Country Link
CN (1) CN114820465B (zh)
WO (1) WO2023193401A1 (zh)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117173692A (zh) * 2023-11-02 2023-12-05 安徽蔚来智驾科技有限公司 3d目标检测方法、电子设备、介质以及驾驶设备
CN117387581A (zh) * 2023-12-12 2024-01-12 北京麦麦趣耕科技有限公司 油樟生物量估算方法、装置以及存储介质
CN117408999A (zh) * 2023-12-13 2024-01-16 安格利(成都)仪器设备有限公司 一种利用点云补全自动检测容器和管道腐蚀坑的方法
CN117826116A (zh) * 2024-03-04 2024-04-05 广东电网有限责任公司中山供电局 双柱水平旋转式隔离开关分合闸状态的确定方法和装置
CN118212368A (zh) * 2024-05-22 2024-06-18 堆知(北京)科技集团有限公司 一种基于不完整轮廓点云数据重构三维图像的方法及系统

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114820465B (zh) * 2022-04-06 2024-04-26 合众新能源汽车股份有限公司 点云检测模型训练方法、装置、电子设备及存储介质
CN115880536B (zh) * 2023-02-15 2023-09-01 北京百度网讯科技有限公司 数据处理方法、训练方法、目标对象检测方法及装置
CN116311172B (zh) * 2023-05-17 2023-09-22 九识(苏州)智能科技有限公司 3d目标检测模型的训练方法、装置、设备及存储介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108734193A (zh) * 2018-03-27 2018-11-02 合肥麟图信息科技有限公司 一种深度学习模型的训练方法及装置
CN111612891A (zh) * 2020-05-22 2020-09-01 北京京东乾石科技有限公司 模型生成方法、点云数据处理方法、装置、设备及介质
CN113553943A (zh) * 2021-07-19 2021-10-26 江苏共知自动化科技有限公司 目标实时检测方法以及装置、存储介质、电子装置
CN113807448A (zh) * 2021-09-23 2021-12-17 广州极飞科技股份有限公司 点云数据分割模型训练方法、装置、存储介质及电子设备
CN114820465A (zh) * 2022-04-06 2022-07-29 合众新能源汽车有限公司 点云检测模型训练方法、装置、电子设备及存储介质

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109784333B (zh) * 2019-01-22 2021-09-28 中国科学院自动化研究所 基于点云带权通道特征的三维目标检测方法及系统
CN110032962B (zh) * 2019-04-03 2022-07-08 腾讯科技（深圳）有限公司 一种物体检测方法、装置、网络设备和存储介质
CN113970922B (zh) * 2020-07-22 2024-06-14 商汤集团有限公司 点云数据的处理方法、智能行驶控制方法及装置
CN113327279B (zh) * 2021-08-04 2021-09-28 腾讯科技（深圳）有限公司 一种点云数据处理方法、装置、计算机设备及存储介质
CN113850916A (zh) * 2021-09-26 2021-12-28 浪潮电子信息产业股份有限公司 模型训练、点云缺失补全方法、装置、设备及介质
CN113902897B (zh) * 2021-09-29 2022-08-23 北京百度网讯科技有限公司 目标检测模型的训练、目标检测方法、装置、设备和介质
CN114005110B (zh) * 2021-12-30 2022-05-17 智道网联科技(北京)有限公司 3d检测模型训练方法与装置、3d检测方法与装置

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108734193A (zh) * 2018-03-27 2018-11-02 合肥麟图信息科技有限公司 一种深度学习模型的训练方法及装置
CN111612891A (zh) * 2020-05-22 2020-09-01 北京京东乾石科技有限公司 模型生成方法、点云数据处理方法、装置、设备及介质
CN113553943A (zh) * 2021-07-19 2021-10-26 江苏共知自动化科技有限公司 目标实时检测方法以及装置、存储介质、电子装置
CN113807448A (zh) * 2021-09-23 2021-12-17 广州极飞科技股份有限公司 点云数据分割模型训练方法、装置、存储介质及电子设备
CN114820465A (zh) * 2022-04-06 2022-07-29 合众新能源汽车有限公司 点云检测模型训练方法、装置、电子设备及存储介质

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117173692A (zh) * 2023-11-02 2023-12-05 安徽蔚来智驾科技有限公司 3d目标检测方法、电子设备、介质以及驾驶设备
CN117173692B (zh) * 2023-11-02 2024-02-02 安徽蔚来智驾科技有限公司 3d目标检测方法、电子设备、介质以及驾驶设备
CN117387581A (zh) * 2023-12-12 2024-01-12 北京麦麦趣耕科技有限公司 油樟生物量估算方法、装置以及存储介质
CN117387581B (zh) * 2023-12-12 2024-02-13 北京麦麦趣耕科技有限公司 油樟生物量估算方法、装置以及存储介质
CN117408999A (zh) * 2023-12-13 2024-01-16 安格利(成都)仪器设备有限公司 一种利用点云补全自动检测容器和管道腐蚀坑的方法
CN117408999B (zh) * 2023-12-13 2024-02-20 安格利(成都)仪器设备有限公司 一种利用点云补全自动检测容器和管道腐蚀坑的方法
CN117826116A (zh) * 2024-03-04 2024-04-05 广东电网有限责任公司中山供电局 双柱水平旋转式隔离开关分合闸状态的确定方法和装置
CN117826116B (zh) * 2024-03-04 2024-05-14 广东电网有限责任公司中山供电局 双柱水平旋转式隔离开关分合闸状态的确定方法和装置
CN118212368A (zh) * 2024-05-22 2024-06-18 堆知(北京)科技集团有限公司 一种基于不完整轮廓点云数据重构三维图像的方法及系统

Also Published As

Publication number Publication date
CN114820465B (zh) 2024-04-26
CN114820465A (zh) 2022-07-29

Similar Documents

Publication Publication Date Title
WO2023193401A1 (zh) 点云检测模型训练方法、装置、电子设备及存储介质
CN110378381B (zh) 物体检测方法、装置和计算机存储介质
WO2022126377A1 (zh) 检测车道线的方法、装置、终端设备及可读存储介质
CN111027493B (zh) 一种基于深度学习多网络软融合的行人检测方法
CN109086811B (zh) 多标签图像分类方法、装置及电子设备
CN107784288B (zh) 一种基于深度神经网络的迭代定位式人脸检测方法
CN110334762B (zh) 一种基于四叉树结合orb和sift的特征匹配方法
CN113076871A (zh) 一种基于目标遮挡补偿的鱼群自动检测方法
CN114359851A (zh) 无人驾驶目标检测方法、装置、设备及介质
WO2023193400A1 (zh) 点云检测和分割方法、装置，以及，电子设备
CN110852327A (zh) 图像处理方法、装置、电子设备及存储介质
CN114973057B (zh) 基于人工智能的视频图像检测方法及相关设备
CN111461145A (zh) 一种基于卷积神经网络进行目标检测的方法
CN115631112B (zh) 一种基于深度学习的建筑轮廓矫正方法及装置
CN115049945B (zh) 一种基于无人机图像的小麦倒伏面积提取方法和装置
CN112801107A (zh) 一种图像分割方法和电子设备
CN116246119A (zh) 3d目标检测方法、电子设备及存储介质
WO2022100607A1 (zh) 一种神经网络结构确定方法及其装置
CN115240240A (zh) 基于yolo网络的红外人脸识别方法及系统
CN111126296A (zh) 水果定位方法及装置
CN114972492A (zh) 一种基于鸟瞰图的位姿确定方法、设备和计算机存储介质
CN111738069A (zh) 人脸检测方法、装置、电子设备及存储介质
Elashry et al. Feature matching enhancement using the graph neural network (gnn-ransac)
CN116486166A (zh) 一种基于边缘计算的输电线异物识别检测方法
CN116012393A (zh) 一种纸箱点云分割方法、装置以及处理设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22936329

Country of ref document: EP

Kind code of ref document: A1