WO2023045252A1 - 模型训练、点云缺失补全方法、装置、设备及介质 - Google Patents

模型训练、点云缺失补全方法、装置、设备及介质 Download PDF

Info

Publication number
WO2023045252A1
WO2023045252A1 PCT/CN2022/078359 CN2022078359W WO2023045252A1 WO 2023045252 A1 WO2023045252 A1 WO 2023045252A1 CN 2022078359 W CN2022078359 W CN 2022078359W WO 2023045252 A1 WO2023045252 A1 WO 2023045252A1
Authority
WO
WIPO (PCT)
Prior art keywords
point cloud
training
cloud data
missing
network
Prior art date
Application number
PCT/CN2022/078359
Other languages
English (en)
French (fr)
Inventor
卢丽华
魏辉
李茹杨
赵雅倩
李仁刚
Original Assignee
浪潮电子信息产业股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 浪潮电子信息产业股份有限公司 filed Critical 浪潮电子信息产业股份有限公司
Publication of WO2023045252A1 publication Critical patent/WO2023045252A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G06T17/20—Finite element generation, e.g. wire-frame surface description, tesselation
    • G06T17/205—Re-meshing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/77—Retouching; Inpainting; Scratch removal

Definitions

  • the present application relates to the field of computer technology, and in particular to a model training, point cloud missing completion method, device, electronic equipment, and computer-readable storage medium.
  • Three-dimensional reconstruction technology reconstructs three-dimensional objects in the virtual world, which is the basis for the realization of three-dimensional vision technologies such as VR/AR (Virtual Reality, virtual reality/Augmented Reality, augmented reality).
  • VR/AR Virtual Reality, virtual reality/Augmented Reality, augmented reality
  • 3D point cloud has become the mainstream representation of 3D reconstruction results.
  • point clouds due to mutual occlusion between objects, technical limitations of hardware equipment, etc., the 3D reconstruction results based on point clouds are missing holes or shape structures.
  • some research work has proposed a point-based completion method, which can directly process point cloud data and obtain point cloud features, and predict complete 3D point clouds or missing point clouds through fully connected or folded decoders, and repair and complete 3D Reconstruction results.
  • the direct input of point cloud reduces the amount of input data and the parameter scale of the neural network, which greatly improves the network training speed.
  • related technologies extract features from the missing input point cloud and obtain the feature representation of the input point cloud, which can only repair data from the perspective of existing data, thereby reducing the accuracy of the complementary data generated by the model.
  • the purpose of this application is to provide model training, point cloud missing complement method, device, electronic equipment and computer-readable storage medium, which improves the accuracy of the processed point cloud data after the completion process.
  • model training method including:
  • the initial model is a point cloud completion model
  • the initial model includes a target reconstruction network and an initial generation network
  • the target reconstruction network includes a target encoding network
  • the target encoding network uses the training missing point cloud data for comparative learning
  • the training missing point cloud The data is input into the target encoding network to obtain input features
  • the input features are input into the initial generation network to obtain missing point cloud data
  • the missing point cloud data is used to generate the training and repairing point cloud data.
  • the generation process of the initial model includes:
  • the initial model is obtained by combining the target reconstruction network with the initial generation network.
  • the learning and training of the initial reconstruction network by using the training missing point cloud data to obtain the target reconstruction network includes:
  • the training missing point cloud data into the initial reconstruction network to obtain target data; wherein the target data includes the input features and reconstructed point cloud data;
  • the initial reconstructed network is determined to be the target reconstructed network.
  • the inputting the training missing point cloud data into the initial reconstruction network to obtain target data includes:
  • the parameter adjustment of the initial reconstruction network by using the comparative learning loss value and the reconstruction loss value includes:
  • the initial encoding network includes several feature extraction blocks, each of which includes a multi-layer perceptron and a down-sampling layer based on the farthest point sampling; the initial decoding network includes multiple multi-layer perceptrons Layer Perceptron and multiple upsampling layers.
  • the acquisition of missing point cloud data for training includes:
  • Each of the original missing point clouds is subjected to different degrees of missing processing to obtain the training missing point cloud data; the missing processing is clipping processing.
  • the initial generation network includes a missing point cloud generation network and a correction network
  • the generation process of the training repair point cloud data includes:
  • the missing point cloud generation network includes a missing point cloud modulation module and a folding decoding module, and the inputting the input feature into the missing point cloud generating network to obtain the missing point cloud data includes:
  • the adjusting the parameters of the initial model based on the training repair point cloud data and the original point cloud data corresponding to the training missing point cloud data includes:
  • the missing point cloud true value data is the difference data between the training missing point cloud data and the corresponding original point cloud data.
  • This application also provides a point cloud missing complement method, including:
  • the application also provides a model training device, comprising:
  • the first obtaining module is used to obtain training missing point cloud data
  • a training module configured to input the training missing point cloud data into the initial model to obtain training and repairing point cloud data, and adjust the The parameters of the initial model
  • a determining module configured to determine that the initial model is a point cloud completion model if it is detected that the training completion condition is met;
  • the initial model includes a target reconstruction network and an initial generation network
  • the target reconstruction network includes a target encoding network
  • the target encoding network uses the training missing point cloud data for comparative learning
  • the training missing point cloud The data is input into the target encoding network to obtain input features
  • the input features are input into the initial generation network to obtain missing point cloud data
  • the missing point cloud data is used to generate the training and repairing point cloud data.
  • the present application also provides a point cloud missing complement device, including:
  • the second acquisition module is used to acquire point cloud data to be completed
  • the completion processing module is configured to input the point cloud data to be completed into the above-mentioned point cloud completion model to obtain processed point cloud data.
  • the present application also provides an electronic device, including a memory and a processor, wherein:
  • the memory is used to store computer programs
  • the processor is configured to execute the computer program, so as to realize the above-mentioned model training method, and/or, the above-mentioned point cloud missing completion method.
  • the present application also provides a computer-readable storage medium for storing a computer program, wherein, when the computer program is executed by a processor, the above-mentioned model training method and/or the above-mentioned point cloud missing completion method are implemented.
  • the model training method obtaineds training missing point cloud data; inputs the training missing point cloud data into the initial model to obtain training and repairing point cloud data, and based on the training repairing point cloud data and the original point cloud corresponding to the training missing point cloud data
  • the data adjusts the parameters of the initial model; if it is detected that the training completion condition is met, the initial model is determined to be a point cloud completion model; wherein, the initial model includes the target reconstruction network and the initial generation network, and the target reconstruction network includes the target encoding network, and the target
  • the encoding network uses the training missing point cloud data for comparative learning, the training missing point cloud data is input into the target encoding network to obtain input features, the input features are input into the initial generation network to obtain missing point cloud data, and the missing point cloud data is used to generate training and repairing point cloud data.
  • the initial model includes the target reconstruction network and the initial generation network, wherein the target reconstruction network can take a certain training missing point cloud data as an anchor point, and learn from other training missing point clouds with different missing conditions.
  • the global structure of the point cloud learned by the network can contain information from different local regions, and then more accurate feature extraction can be performed.
  • the initial generation network is used to generate the missing point cloud data, which infers the missing point cloud part lost in the training missing point cloud data based on the input features corresponding to the training missing point cloud data.
  • the missing point cloud features are learned from the input features.
  • the initial model satisfies the training completion condition, it is determined as the point cloud completion model.
  • the point cloud completion model can obtain the global structure with local area information, and accurately predict the missing point cloud according to the input data, thereby improving the accuracy of the processed point cloud data after the completion process, and solving the problem of related problems.
  • the technology has the problem of low data accuracy.
  • the present application also provides a point cloud missing complement method, a model training device, a point cloud missing completing device, an electronic device, and a computer-readable storage medium, which also have the above beneficial effects.
  • Fig. 1 is a flow chart of a model training method provided by the embodiment of the present application.
  • FIG. 2 is a structural diagram of a specific point cloud completion model provided by the embodiment of the present application.
  • FIG. 3 is a schematic structural diagram of a model training device provided in an embodiment of the present application.
  • FIG. 4 is a schematic structural diagram of a point cloud missing complement device provided by an embodiment of the present application.
  • FIG. 5 is a schematic structural diagram of an electronic device provided by an embodiment of the present application.
  • FIG. 1 is a flow chart of a model training method provided in an embodiment of the present application.
  • the method includes:
  • Training missing point cloud data refers to incomplete 3D point cloud data used for model training.
  • Each training missing point cloud data corresponds to an original point cloud data.
  • the original point cloud data can be used as label data in the training process. It can calculate the loss value so that the trained model can recognize the difference between the two, and then learn the ability to predict the missing part of the incomplete 3D point cloud.
  • the training missing point cloud data and its corresponding original point cloud data can be obtained from the existing data set, the original point cloud data is usually the true value of the point cloud Data, that is, the complete point cloud data of an object.
  • the model is obtained by point cloud completion. There is a certain difference with the real result.
  • the missing three-dimensional point cloud data can be used as the original point cloud data, and further missing processing is performed on it to obtain training missing point cloud data. Specifically, this method can include follows the steps below:
  • Step 11 Obtain several original missing point clouds.
  • Step 12 Perform different degrees of missing processing on each original missing point cloud to obtain training missing point cloud data.
  • the original missing point cloud refers to incomplete three-dimensional point cloud data as the original point cloud data, and this embodiment does not limit its specific number.
  • Missing processing refers to the processing that causes the incompleteness of the 3D point cloud data, specifically, it can be cropping processing. Cropping processing, that is, selecting part of the original missing point cloud for deletion. It is understandable that the cropping process causes the loss of some content in the original missing point cloud, resulting in training missing point cloud data with a higher degree of damage than the original missing point cloud. It can be understood that after an original missing point cloud undergoes different degrees of missing processing, corresponding multiple training missing point cloud data can be obtained, and the specific form of each training missing point cloud data is related to the degree of missing processing.
  • the missing point cloud data when the missing point cloud data is generated by the missing processing, the missing content of the training missing point cloud data and the original missing point cloud can be clarified, and the missing content can be called missing true value data.
  • the initial model can not only extract features from incomplete 3D point clouds, learn its global structure, but also predict missing point clouds (that is, missing parts) to obtain predicted missing point clouds. Therefore, in one embodiment, the missing true value data can also be used as label data for a certain part of the initial model during training, so that the model can make accurate predictions.
  • S102 Input the training missing point cloud data into the initial model to obtain training and repairing point cloud data, and adjust the initial model based on the training and repairing point cloud data and the original point cloud data corresponding to the training missing point cloud data parameter.
  • the specific content and form of the training completion condition are not limited, for example, it may be a training round condition, or it may be a training time condition, or it may be a model accuracy condition, or it may be any other optional condition.
  • the initial model includes two parts: the target reconstruction network and the initial generation network.
  • the target reconstruction network refers to the network used to at least extract features from the training missing point cloud data.
  • the target reconstruction network can also perform data reconstruction according to the extracted features, and remove the noise in the training missing point cloud data through data reconstruction.
  • the target reconstruction network includes a target encoding network
  • the target encoding network refers to a network for feature extraction. It can be understood that if the target reconstruction network does not perform the step of data reconstruction, then the target reconstruction network is the target encoding network.
  • the initial generation network refers to the network that generates missing point cloud data and uses it to generate training and repair point cloud data.
  • the data part and the label part of the training data used in model training correspond one-to-one.
  • the model can only learn the global structure of the training data from the perspective of the overall situation of the data part, and based on the global structure Perform feature extraction. Acquisition of this global structure depends on how missing the data part is compared to the label part, and thus is usually not accurate enough. In order to obtain a better global structure, and then obtain input features that can more accurately reflect the training missing point cloud data.
  • the target encoding network uses certain training missing point cloud data as anchor point clouds for comparative learning.
  • the anchor point cloud refers to the point cloud used as the learning benchmark for comparison learning.
  • the training missing point cloud data corresponding to the same original point cloud data as the anchor point cloud is a positive sample, and the training missing point corresponding to other original point cloud data Cloud data is a negative sample.
  • the corresponding input features can be obtained.
  • the input feature is input into the initial generation network to obtain the missing point cloud data, and the missing point cloud data is used to generate training and repairing point cloud data.
  • the initial model in order to improve the convergence speed of the initial model, can be formed by using the pre-trained target reconstruction network.
  • the pre-training will make the parameters of the target reconstruction network basically determined.
  • the generation process of the initial model includes:
  • Step 21 Use the training missing point cloud data to perform comparative learning training on the initial reconstructed network to obtain the target reconstructed network.
  • Step 22 Use the combination of the target reconstruction network and the initial generation network to obtain the initial model.
  • the initial reconstruction network refers to the reconstruction network that has not been trained. It can be pre-trained by using the missing point cloud data.
  • the pre-training is also the comparison learning training to obtain the target reconstruction network.
  • the initial model is obtained by combining the target reconstruction network with the initial generation network. In the subsequent training of the initial model, since the target reconstruction network has been pre-trained, it basically achieves convergence. Therefore, compared with the scheme of using a completely untrained initial reconstruction network as the target reconstruction network to form an initial model, it has a pre-trained The target initial model can reach convergence faster.
  • the process of using the training missing point cloud data to perform comparative learning and training on the initial reconstructed network to obtain the target reconstructed network may include the following steps:
  • Step 31 Determine the anchor point cloud from the training missing point cloud data.
  • Step 32 Based on the anchor point cloud, input the training missing point cloud data into the initial reconstruction network to obtain the target data.
  • Step 33 Use the input features to obtain the comparative learning loss value, use the reconstructed point cloud data to obtain the reconstruction loss value, and use the comparative learning loss value and the reconstruction loss value to adjust the parameters of the initial reconstruction network.
  • Step 34 If it is detected that the pre-training completion condition is satisfied, then determine the initial reconstructed network as the target reconstructed network.
  • the initial reconstruction network not only extracts features from the input training missing point cloud data, but also performs data reconstruction based on the extracted features, so as to remove noise in the training missing point cloud data. Therefore, the target data includes input features and reconstructed point cloud data.
  • the input feature refers to the feature obtained after feature extraction of the input training missing point cloud data; the reconstructed point cloud data refers to the reconstructed data obtained after data reconstruction using the input feature.
  • P in can be used to represent the original point cloud data
  • S in can be used to represent the set of original point cloud data
  • S S can be used to represent the set of training missing point cloud data.
  • the training missing point cloud data corresponding to the aircraft in S S (that is, each missing aircraft point cloud obtained according to the aircraft point cloud) is positive Samples, training missing point cloud data that do not correspond to airplanes (such as chair point clouds with missing points) are negative samples. It can be understood that when a positive sample or a negative sample is input into the initial reconstruction network, the corresponding sample type (ie positive sample or negative sample) needs to be declared.
  • the corresponding loss values are calculated using the input features and the reconstructed point cloud data respectively.
  • the comparative learning loss value is obtained by using the input features, and the comparative learning loss value refers to the loss value used to adjust the parameters of the feature extraction part;
  • the reconstruction loss value is calculated by using the reconstructed point cloud data, and the reconstruction loss value refers to The loss value used for parameter tuning of the data reconstruction part.
  • the process of inputting the training missing point cloud data into the initial reconstruction network to obtain the target data may include the following steps:
  • Step 41 Input the training missing point cloud data into the initial encoding network in the initial reconstruction network to obtain input features.
  • Step 42 Input the input features into the initial decoding network in the initial reconstruction network to obtain reconstructed point cloud data.
  • the process of adjusting the parameters of the initial reconstruction network by using the comparative learning loss value and the reconstruction loss value may include the following steps:
  • Step 43 Generate a first loss value by using the comparative learning loss value and the reconstruction loss value.
  • Step 44 Use the first loss value to adjust the parameters of the initial reconstructed network.
  • the initial reconstruction network includes an initial encoding network and an initial decoding network
  • the target encoding network is used to extract features from training missing point cloud data to obtain input features.
  • the initial decoding network is used to decode the input features in order to complete data reconstruction and obtain reconstructed point cloud data.
  • the first loss value can be obtained, and then the parameters of the entire initial reconstruction network can be adjusted by using the first loss value.
  • This embodiment does not limit the specific manner of generating the first loss value. For example, in an implementation manner, the two may be added to obtain the first loss value.
  • the initial encoding network can use PointNet++ (a network structure for processing point clouds) as the basic framework, which includes several feature extraction blocks, each feature extraction block includes an MLP (Multi-layer Perceptron, multilayer perceptron) and a downsampling layer.
  • MLP is used to optimize the extracted point cloud features
  • FPS Field Point Sampling, the farthest point sampling
  • the pooling layer in the initial encoding network is used to perform pooling processing to obtain the global features of the point cloud.
  • the initial decoding network including multiple MLPs for feature dimension transformation and upsampling with multiple upsampling layers, can iteratively reconstruct the shape of the input point cloud.
  • the combination of the initial encoding network and the initial decoding network of this structure can better remove the noise in the input point cloud and optimize the shape of the input point cloud.
  • each training missing point cloud data has the same global structure.
  • different training missing point cloud data as different local parts of the same original point cloud data, has a limited receptive field.
  • Using contrastive learning to train the initial encoding network can make the global structure of the point cloud learned by the initial encoding network contain Information from different local areas.
  • the above process is illustrated by an example: input the missing point cloud data of the category "aircraft" into the initial reconstruction network, and the initial encoding network can obtain the local detail features representing each part of the aircraft, as well as the global structural features representing the whole, that is, the global structure .
  • the global structure of the positive and negative samples of comparative learning is obtained, and used as the input of the initial decoding network to obtain the reconstructed point cloud data.
  • the calculation minimizes the contrastive learning loss and the reconstruction loss, and uses them to update the network parameters, and then continuously optimizes the local and global features extracted from the input point cloud.
  • L i NCE can be used to represent the comparative learning loss value
  • L in can be used to represent the reconstruction loss value
  • InfoNCE loss can be used as the loss function of the comparative learning loss value.
  • the specific calculation formula is:
  • v represents the feature of the anchor point cloud
  • v+ represents the input feature of the positive sample
  • v- represents the input feature of the negative sample
  • ⁇ is a constant.
  • the initial generation network can be directly spliced according to the generated missing point cloud data and training missing point cloud data (or reconstructed point cloud data obtained after reconstruction) to obtain training repair points cloud data.
  • the data obtained by direct splicing can be converted into rough 3D point cloud data, and the initial generation network can further optimize the rough 3D point cloud data to obtain training and repairing point cloud data.
  • the initial generation network includes a missing point cloud generation network and a correction network, and the generation process of training and repairing point cloud data may include the following steps:
  • Step 51 Input the input features into the missing point cloud generation network to obtain missing point cloud data.
  • Step 52 Input the missing point cloud data and the output data output by the target reconstruction network into the correction network to obtain training and repair point cloud data.
  • the missing point cloud generation network refers to a network for generating corresponding missing point cloud data according to input features.
  • the corrected network refers to a network that performs shape correction on the output data (which can be unrestructured training missing point cloud data, or reconstructed reconstructed point cloud data).
  • the specific structures of the missing point cloud generation network and the correction network are not limited, and can be set as required.
  • the missing point cloud generation network includes a missing point cloud modulation module and a folding decoding module
  • the process of inputting input features into the missing point cloud generation network to obtain missing point cloud data may include the following steps:
  • Step 53 Input the input features into the missing point cloud modulation module to obtain the missing point cloud features.
  • Step 54 Input missing point cloud features and input features into the folding and decoding module to obtain missing point cloud data.
  • the missing point cloud generation network includes multiple decoding modules, and each decoding module contains a missing point cloud modulation module and a folding-based decoding layer (ie, a folding decoding module).
  • the missing point cloud modulation module transforms the input features through an MLP as the learned missing point cloud features.
  • the randomly sampled 2D grid Based on the folded decoding layer, the randomly sampled 2D grid, the learned missing point cloud features and the input features are processed to obtain the missing point cloud data. By increasing the density of the 2D mesh layer by layer, higher resolution missing point clouds can be predicted.
  • This embodiment does not limit the specific process for the correction network to obtain training and repair point cloud data.
  • the correction network uses FPS sampling to obtain a rough 3D point cloud.
  • the correction network includes multiple MLPs and a fold-based correction layer.
  • the point cloud features can be obtained, and then the 2D grid is randomly sampled from a fixed-size 2D plane.
  • the sampled 2D grid, point cloud features, and 3D coordinates of the point cloud are input into the correction layer based on folding, and the rough 3D point cloud is optimized by using it to obtain training repair point cloud data.
  • the process of adjusting the parameters of the initial model based on the training repaired point cloud data and the training missing point cloud data may include the following steps:
  • Step 61 Use the training repair point cloud data and the original point cloud data to obtain the corrected reconstruction loss value.
  • Step 62 Using the missing point cloud data and the missing point cloud ground truth data to obtain the missing reconstruction loss value.
  • Step 63 Generate a second loss value by using the corrected reconstruction loss value and the missing reconstruction loss value.
  • Step 64 Use the second loss value to adjust the parameters of the initial model.
  • the missing true value data is the difference data between the training missing point cloud data and the corresponding original point cloud data.
  • L r can be used to represent the corrected reconstruction loss value
  • L c can be used to represent the missing reconstruction loss value.
  • L r and L c are calculated in the same way as Lin .
  • FIG. 2 is a structural diagram of a specific point cloud completion model provided by the embodiment of the present application.
  • the incomplete 3D point cloud is the missing point cloud data for training or the input point cloud data to be completed when the model is trained and used.
  • the input point cloud reconstruction network based on comparative learning is the target reconstruction network.
  • the decoding modulation network is the missing point cloud generation network
  • the rough point cloud prediction correction network is the correction network.
  • module 1 is the target encoding network (or initial encoding network), which is used for feature encoding
  • module 2 is the initial decoding network, which is used for fully connected decoding
  • module 3 is the folding decoding module, which is used for folding decoding
  • module 4 It is the correction network, which is used for rough point cloud correction
  • module 5 is the missing point cloud modulation module, which is used for missing point cloud modulation and generating missing point cloud features.
  • the application also provides a method for complementing the missing point cloud, which may include the following steps:
  • Step 71 Obtain point cloud data to be completed.
  • Step 72 Input the point cloud data to be completed into the above-mentioned point cloud completion model to obtain the processed point cloud data.
  • the initial model includes the target reconstruction network and the initial generation network, wherein the target reconstruction network can use the original point cloud data as the anchor point, from the training missing point cloud with different missing situations
  • the global structure of the point cloud learned by the network can contain information from different local regions, and then more accurate feature extraction can be performed.
  • the initial generation network is used to generate the missing point cloud data, which infers the missing point cloud part lost in the training missing point cloud data based on the input features corresponding to the training missing point cloud data.
  • the missing point cloud features are learned from the input features.
  • the initial model satisfies the training completion condition, it is determined as the point cloud completion model.
  • the point cloud completion model can obtain the global structure with local area information, and accurately predict the missing point cloud according to the input data, thereby improving the accuracy of the processed point cloud data after the completion process, and solving the problem of related problems.
  • Technology has the problem of low data accuracy.
  • model training device provided by the embodiment of the present application is introduced below, and the model training device described below and the model training method described above can be referred to in correspondence.
  • Figure 3 is a schematic structural diagram of a model training device provided in the embodiment of the present application, including:
  • the first obtaining module 110 is used to obtain training missing point cloud data
  • the training module 120 is used to input the missing point cloud data into the initial model for training, obtain the training repair point cloud data, and adjust the parameters of the initial model based on the original point cloud data corresponding to the training repair point cloud data and the training missing point cloud data;
  • Determining module 130 is used for determining that initial model is the point cloud completion model if detecting that the training completion condition is met;
  • the initial model includes the target reconstruction network and the initial generation network.
  • the target reconstruction network includes the target encoding network.
  • the target encoding network uses the training missing point cloud data for comparative learning, and the training missing point cloud data is input into the target encoding network to obtain the input features.
  • Input The feature input initial generation network obtains the missing point cloud data, and the missing point cloud data is used to generate training and repairing point cloud data.
  • the pre-training module is used to learn and train the initial reconstruction network by using the training missing point cloud data to obtain the target reconstruction network;
  • the combination module is used to combine the target reconstruction network and the initial generation network to obtain an initial model.
  • the pre-training module includes:
  • Anchor point determining unit for determining anchor point cloud from training missing point cloud data
  • the input unit is used to input the training missing point cloud data into the initial reconstruction network based on the anchor point cloud to obtain target data; wherein, the target data includes input features and reconstructed point cloud data;
  • the parameter adjustment unit is used to obtain a comparative learning loss value by using the input feature, obtain a reconstruction loss value by using the reconstructed point cloud data, and use the comparative learning loss value and the reconstruction loss value to adjust the parameters of the initial reconstruction network;
  • the target reconstructed network determination unit is configured to determine the initial reconstructed network as the target reconstructed network if it is detected that the pre-training completion condition is met.
  • the input unit includes:
  • the feature acquisition subunit is used to input the training missing point cloud data into the initial encoding network in the initial reconstruction network to obtain input features;
  • the reconstruction subunit is used to input the input features into the initial decoding network in the initial reconstruction network to obtain reconstructed point cloud data;
  • the parameter adjustment unit includes:
  • the first loss generation subunit is used to generate the first loss value by using the contrastive learning loss value and the reconstruction loss value;
  • the initial reconstructed network adjustment subunit is configured to use the first loss value to adjust the parameters of the initial reconstructed network.
  • the initial encoding network includes several feature extraction blocks, each feature extraction block includes a multi-layer perceptron and a down-sampling layer based on the farthest point sampling; the initial decoding network includes multiple multi-layer perceptrons and multiple Upsampling layer.
  • the first acquisition module 110 includes:
  • the original missing acquisition unit is used to obtain several original missing point clouds as the original point cloud data;
  • the missing processing unit is used to perform different degrees of missing processing on each original missing point cloud to obtain training missing point cloud data; the missing processing is clipping processing.
  • the training module 120 includes:
  • the missing point cloud generation unit inputs the input features into the missing point cloud generation network to obtain missing point cloud data
  • the correction unit is used to input the missing point cloud data and the output data output by the target reconstruction network into the correction network to obtain training and repairing point cloud data;
  • the missing point cloud generation network includes the missing point cloud modulation module and the folding decoding module, and the missing point cloud generation unit includes:
  • the missing feature acquisition subunit is used to input the input feature into the missing point cloud modulation module to obtain the missing point cloud feature;
  • the folding decoding subunit is used to input missing point cloud features and input features into the folding decoding module to obtain missing point cloud data.
  • the training module 120 includes:
  • a modified reconstruction loss generation unit is used to obtain a modified reconstruction loss value using the training repair point cloud data and the original point cloud data;
  • the missing reconstruction loss generation unit is used to obtain the missing reconstruction loss value by using the missing point cloud data and the missing point cloud true value data;
  • a second loss generation unit configured to generate a second loss value using the corrected reconstruction loss value and the missing reconstruction loss value
  • the initial model adjustment unit is configured to adjust the parameters of the initial model by using the second loss value.
  • point cloud missing complement device provided by the embodiment of the present application.
  • the point cloud missing complement device described below and the point cloud missing complement method described above can be referred to in correspondence.
  • FIG. 4 is a schematic structural diagram of a point cloud missing complement device provided in an embodiment of the present application, including:
  • the second obtaining module 210 is used to obtain point cloud data to be completed
  • the completion processing module 220 is configured to input the point cloud data to be completed into the above-mentioned point cloud completion model to obtain processed point cloud data.
  • the electronic device provided by the embodiment of the present application is introduced below, and the electronic device described below and the model training method described above may be referred to in correspondence.
  • the electronic device 100 may include a processor 101 and a memory 102 , and may further include one or more of a multimedia component 103 , an information input/information output (I/O) interface 104 and a communication component 105 .
  • a multimedia component 103 may be included in the electronic device 100 .
  • I/O information input/information output
  • the processor 101 is used to control the overall operation of the electronic device 100, so as to complete all or part of the steps in the above-mentioned model training method;
  • the memory 102 is used to store various types of data to support the operation of the electronic device 100, these data For example, instructions for any application or method operating on the electronic device 100 may be included, as well as application-related data.
  • the memory 102 can be realized by any type of volatile or non-volatile storage device or their combination, such as Static Random Access Memory (Static Random Access Memory, SRAM), Electrically Erasable Programmable Read-Only Memory (Electrically Erasable Programmable Read-Only Memory, EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (Read-Only Memory, One or more of Only Memory, ROM), magnetic memory, flash memory, magnetic disk or optical disk.
  • Static Random Access Memory Static Random Access Memory
  • SRAM Static Random Access Memory
  • EEPROM Electrically Erasable Programmable Read-Only Memory
  • EPROM Erasable Programmable Read-Only Memory
  • PROM Programmable Read-Only Memory
  • Read-Only Memory One or more of Only Memory, ROM
  • magnetic memory flash memory
  • flash memory magnetic disk or optical disk.
  • Multimedia components 103 may include screen and audio components.
  • the screen can be, for example, a touch screen, and the audio component is used for outputting and/or inputting audio signals.
  • an audio component may include a microphone for receiving external audio signals.
  • the received audio signal may be further stored in the memory 102 or sent via the communication component 105 .
  • the audio component also includes at least one speaker for outputting audio signals.
  • the I/O interface 104 provides an interface between the processor 101 and other interface modules, which may be a keyboard, a mouse, buttons, and the like. These buttons can be virtual buttons or physical buttons.
  • the communication component 105 is used for wired or wireless communication between the electronic device 100 and other devices.
  • Wireless communication such as Wi-Fi, Bluetooth, near field communication (Near Field Communication, NFC for short), 2G, 3G or 4G, or a combination of one or more of them, so the corresponding communication component 105 may include: Wi-Fi parts, Bluetooth parts, NFC parts.
  • the electronic device 100 may be implemented by one or more Application Specific Integrated Circuit (ASIC for short), Digital Signal Processor (DSP for short), Digital Signal Processing Device (DSPD for short), Programmable Logic Device (Programmable Logic Device, PLD for short), Field Programmable Gate Array (Field Programmable Gate Array, FPGA for short), controller, microcontroller, microprocessor or other electronic components are implemented for implementing the above embodiments The model training method given.
  • ASIC Application Specific Integrated Circuit
  • DSP Digital Signal Processor
  • DSPD Digital Signal Processing Device
  • PLD Programmable Logic Device
  • Field Programmable Gate Array Field Programmable Gate Array
  • FPGA Field Programmable Gate Array
  • the computer-readable storage medium provided by the embodiment of the present application is introduced below, and the computer-readable storage medium described below and the model training method described above can be referred to in correspondence.
  • the present application also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above-mentioned model training method are implemented.
  • the computer-readable storage medium may include: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk, etc., which can store program codes. medium.
  • each embodiment in this specification is described in a progressive manner, each embodiment focuses on the difference from other embodiments, and the same or similar parts of each embodiment can be referred to each other.
  • the description is relatively simple, and for the related information, please refer to the description of the method part.
  • RAM random access memory
  • ROM read-only memory
  • EEPROM electrically programmable ROM
  • EEPROM electrically erasable programmable ROM
  • registers hard disk, removable disk, CD-ROM, or any other Any other known storage medium.

Landscapes

  • Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Graphics (AREA)
  • Geometry (AREA)
  • Software Systems (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

本申请公开了模型训练、点云缺失补全方法、装置、电子设备及计算机可读存储介质，模型训练方法包括：获取训练缺失点云数据；将训练缺失点云数据输入初始模型，得到训练修复点云数据，并基于训练修复点云数据和训练缺失点云数据对应的原始点云数据调整初始模型的参数；若检测到满足训练完成条件，则确定初始模型为点云补全模型；其中，初始模型包括目标重构网络和初始生成网络，目标重构网络包括目标编码网络，目标编码网络利用训练缺失点云数据进行对比学习，训练缺失点云数据输入目标编码网络得到输入特征，输入特征输入初始生成网络得到缺失点云数据，缺失点云数据用于生成训练修复点云数据；提高了补全处理后的处理后点云数据的准确性。

Description

模型训练、点云缺失补全方法、装置、设备及介质
本申请要求在2021年9月26日提交中国专利局、申请号为202111129999.6、发明名称为“模型训练、点云缺失补全方法、装置、设备及介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及计算机技术领域，特别涉及模型训练、点云缺失补全方法、装置、电子设备及计算机可读存储介质。
背景技术
三维重建技术将三维的物体在虚拟世界中重建出来，是VR/AR(VirtualReality，虚拟现实/Augmented Reality，增强现实)等三维视觉技术的实现基础。近年来，随着传感器、深度学习等的发展，三维点云已经成为三维重建结果的主流表示方式。但是，由于物体间相互遮挡、硬件设备技术限制等原因，基于点云的三维重建结果存在孔洞或形状结构的缺失。目前，有研究工作提出基于点的补全方法，可以直接处理点云数据并获取点云特征，并通过基于全连接或者折叠的解码器，预测完整三维点云或者缺失点云，修复补全三维重建结果。相比于体素模型表示，直接输入点云减少了输入数据量和神经网络的参数规模，使得网络训练速度得到极大提高。然而，相关技术对存在缺失的输入点云提取特征，得到输入点云的特征表示，仅能从已有数据的角度进行数据修复，从而降低了模型生成的补全数据的准确性。
因此，相关技术存在的数据准确性低的问题，是本领域技术人员需要解决的技术问题。
发明内容
有鉴于此，本申请的目的在于提供模型训练、点云缺失补全方法、装置、电子设备及计算机可读存储介质，提高了补全处理后的处理后点云数据的准确性。
为解决上述技术问题，本申请提供了一种模型训练方法，包括：
获取训练缺失点云数据；
将所述训练缺失点云数据输入初始模型，得到训练修复点云数据，并基于所述训练修复点云数据和所述训练缺失点云数据对应的原始点云数据调整所述初始模型的参数；
若检测到满足训练完成条件，则确定所述初始模型为点云补全模型；
其中，所述初始模型包括目标重构网络和初始生成网络，所述目标重构网络包括目标编码网络，所述目标编码网络利用所述训练缺失点云数据进行对比学习，所述训练缺失点云数据输入所述目标编码网络得到输入特征，所述输入特征输入所述初始生成网络得到缺失点云数据，所述缺失点云数据用于生成所述训练修复点云数据。
可选地，所述初始模型的生成过程，包括：
利用所述训练缺失点云数据对初始重构网络进行学习训练，得到所述目标重构网络；
利用所述目标重构网络与所述初始生成网络组合得到所述初始模型。
可选地，所述利用所述训练缺失点云数据对初始重构网络进行学习训练，得到所述目标重构网络，包括：
从所述训练缺失点云数据中确定锚点点云；
基于所述锚点点云，将所述训练缺失点云数据输入所述初始重构网络，得到目标数据；其中，所述目标数据包括所述输入特征和重构点云数据；
利用所述输入特征得到对比学习损失值，利用所述重构点云数据得到重建损失值，并利用所述对比学习损失值和所述重建损失值对所述初始重构网络进行参数调整；
若检测到满足预训练完成条件，则确定所述初始重构网络为所述目标重构网络。
可选地，所述将所述训练缺失点云数据输入所述初始重构网络，得到目标数据，包括：
将所述训练缺失点云数据输入所述初始重构网络中的初始编码网络，得到所述输入特征；
将所述输入特征输入所述初始重构网络中的初始解码网络，得到所述重构点云数据；
相应的，所述利用所述对比学习损失值和所述重建损失值对所述初始重构网络进行参数调整，包括：
利用所述对比学习损失值和所述重建损失值生成第一损失值；
利用所述第一损失值对所述初始重构网络进行参数调整。
可选地，所述初始编码网络包括若干个特征提取块，每个所述特征提取块包括一个多层感知机和一个基于最远点采样的下采样层；所述初始解码网络包括多个多层感知机和多个上采样层。
可选地，所述获取训练缺失点云数据，包括：
获取若干个原始缺失点云作为所述原始点云数据；
分别对各个所述原始缺失点云进行不同程度的缺失处理，得到所述训练缺失点云数据；所述缺失处理为裁剪处理。
可选地，所述初始生成网络包括缺失点云生成网络和修正网络，所述训练修复点云数据的生成过程包括：
将所述输入特征输入所述缺失点云生成网络得到所述缺失点云数据；
将所述缺失点云数据和所述目标重构网络输出的输出数据输入所述修正网络，得到所述训练修复点云数据；
其中，所述缺失点云生成网络包括缺失点云调制模块和折叠解码模块，所述将所述输入特征输入所述缺失点云生成网络得到所述缺失点云数据，包括：
将所述输入特征输入所述缺失点云调制模块，得到缺失点云特征；
将所述缺失点云特征、所述输入特征输入所述折叠解码模块，得到所述缺失点云数据。
可选地，所述基于所述训练修复点云数据和所述训练缺失点云数据对应的原始点云数据调整所述初始模型的参数，包括：
利用所述训练修复点云数据和原始点云数据得到修正重建损失值；
利用所述缺失点云数据与缺失点云真值数据得到缺失重建损失值；
利用所述修正重建损失值和所述缺失重建损失值生成第二损失值；
利用所述第二损失值对所述初始模型进行参数调整；
其中，所述缺失点云真值数据为所述训练缺失点云数据与对应的原始点云数据的差值数据。
本申请还提供了一种点云缺失补全方法，包括：
获取待补全点云数据；
将所述待补全点云数据输入上述的点云补全模型，得到处理后点云数据。
本申请还提供了一种模型训练装置，包括：
第一获取模块，用于获取训练缺失点云数据；
训练模块，用于将所述训练缺失点云数据输入初始模型，得到训练修复点云数据，并基于所述训练修复点云数据和所述训练缺失点云数据对应的原始点云数据调整所述初始模型的参数；
确定模块，用于若检测到满足训练完成条件，则确定所述初始模型为点云补全模型；
其中，所述初始模型包括目标重构网络和初始生成网络，所述目标重构网络包括目标编码网络，所述目标编码网络利用所述训练缺失点云数据进行对比学习，所述训练缺失点云数据输入所述目标编码网络得到输入特征，所述输入特征输入所述初始生成网络得到缺失点云数据，所述缺失点云数据用于生成所述训练修复点云数据。
本申请还提供了一种点云缺失补全装置，包括：
第二获取模块，用于获取待补全点云数据；
补全处理模块，用于将所述待补全点云数据输入上述的点云补全模型，得到处理后点云数据。
本申请还提供了一种电子设备，包括存储器和处理器，其中：
所述存储器，用于保存计算机程序；
所述处理器，用于执行所述计算机程序，以实现上述的模型训练方法，和/或，上述的点云缺失补全方法。
本申请还提供了一种计算机可读存储介质，用于保存计算机程序，其中，所述计算机程序被处理器执行时实现上述的模型训练方法，和/或，上述的点云缺失补全方法。
本申请提供的模型训练方法，获取训练缺失点云数据；将训练缺失点云数据输入初始模型，得到训练修复点云数据，并基于训练修复点云数据和训练缺失点云数据对应的原始点云数据调整初始模型的参数；若检测到满足训 练完成条件，则确定初始模型为点云补全模型；其中，初始模型包括目标重构网络和初始生成网络，目标重构网络包括目标编码网络，目标编码网络利用训练缺失点云数据进行对比学习，训练缺失点云数据输入目标编码网络得到输入特征，输入特征输入初始生成网络得到缺失点云数据，缺失点云数据用于生成训练修复点云数据。
可见，该方法中，初始模型中包括目标重构网络和初始生成网络，其中，目标重构网络能够以某个训练缺失点云数据为锚点，从具有不同缺失情况的其他的训练缺失点云数据的角度对全局结构进行学习。即对应于同一个原始点云数据的若干个训练缺失点云数据，其具有相同的全局结构，但是由于缺失的部分不同，使得其具有有限且不同的感受野，基于比对学习的训练方式，使得网络学习到的点云全局结构能够包含来自不同局部区域的信息，进而能够进行更准确地特征提取。初始生成网络用于生成缺失点云数据，其基于训练缺失点云数据对应的输入特征对训练缺失点云数据失去的缺失点云部分进行推测。在训练时，根据输入特征学习提取缺失点云特征。在初始模型满足训练完成条件时，将其确定为点云补全模型。点云补全模型能够获取具有局部区域信息的全局结构，并根据输入数据的情况对缺失点云进行准确地预测，进而提高了补全处理后的处理后点云数据的准确性，解决了相关技术存在的数据准确性低的问题。
此外，本申请还提供了点云缺失补全方法、模型训练装置、点云缺失补全装置、电子设备及计算机可读存储介质，同样具有上述有益效果。
附图说明
为了更清楚地说明本申请实施例或相关技术中的技术方案，下面将对实施例或相关技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据提供的附图获得其他的附图。
图1为本申请实施例提供的一种模型训练方法流程图；
图2为本申请实施例提供的一种具体的点云补全模型结构图；
图3为本申请实施例提供的一种模型训练装置的结构示意图；
图4为本申请实施例提供的一种点云缺失补全装置的结构示意图；
图5为本申请实施例提供的一种电子设备的结构示意图。
具体实施方式
为使本申请实施例的目的、技术方案和优点更加清楚，下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
请参考图1，图1为本申请实施例提供的一种模型训练方法流程图。该方法包括：
S101：获取训练缺失点云数据。
训练缺失点云数据，是指用于进行模型训练的不完整的三维点云数据，每个训练缺失点云数据对应于一个原始点云数据，原始点云数据可以作为训练过程的标签数据，利用其可以进行损失值的计算，以便被训练的模型能够识别二者之间的差别，进而学习到预测不完整的三维点云缺失部分的能力。
对于训练缺失点云数据的获取方式，在一种实施方式中，可以从已有的数据集中获取训练缺失点云数据以及其对应的原始点云数据，该原始点云数据通常为点云真值数据，即某一物体的完整的点云数据。在实际应用中，点云真值数据的获取难度较大，数量较少，且通常不够准确，因此利用点云真值数据作为原始点云数据训练模型后，模型进行点云补全得到的结果与真实结果具有一定差异。为了解决上述问题，在另一种实施方式中，可以采用具有缺失的三维点云数据作为原始点云数据，并对其进行进一步缺失处理，得到训练缺失点云数据，具体的，该方式可以包括如下步骤：
步骤11：获取若干个原始缺失点云。
步骤12：分别对各个原始缺失点云进行不同程度的缺失处理，得到训练缺失点云数据。
原始缺失点云，是指作为原始点云数据的不完整的三维点云数据，本实施例并不限定其具体数量。缺失处理，是指造成三维点云数据残缺的处理， 具体可以为裁剪处理。裁剪处理，即在原始缺失点云上选择部分内容删除。可以理解的是，裁剪处理造成原始缺失点云中部分内容的缺失，造成比原始缺失点云残损程度更高的训练缺失点云数据。可以理解的是，一个原始缺失点云经过不同程度的缺失处理后，可以得到对应的多个训练缺失点云数据，各个训练缺失点云数据的具体形式与缺失处理的程度相关。
可以理解的是，缺失处理生成训练缺失点云数据的同时，可以明确训练缺失点云数据与原始缺失点云的缺失内容，该缺失内容可以被称为缺失真值数据。由于本申请中，初始模型不仅能够对不完整三维点云进行特征提取，学习其全局结构，还能够对缺失点云(即缺失的部分)进行预测得到预测的缺失点云。因此在一种实施方式中，缺失真值数据同样可以作为初始模型中某一部分训练时的标签数据，以便模型能够进行准确预测。
S102：将所述训练缺失点云数据输入初始模型，得到训练修复点云数据，并基于所述训练修复点云数据和所述训练缺失点云数据对应的原始点云数据调整所述初始模型的参数。
S103：若检测到满足训练完成条件，则确定初始模型为点云补全模型。
其中，训练完成条件的具体内容和形式不做限定，例如可以为训练轮次条件，或者可以为训练时间条件，或者可以为模型准确度条件，或者可以为其他任意可选的条件。
需要说明的是，初始模型包括目标重构网络和初始生成网络两个部分，目标重构网络，是指至少用于对训练缺失点云数据进行特征提取的网络，除此之外，通常情况下，目标重构网络还可以根据提取到的特征进行数据重构，通过数据重构的方式去除训练缺失点云数据中的噪声。其中，目标重构网络包括目标编码网络，目标编码网络，是指用于进行特征提取的网络。可以理解的是，若目标重构网络并不执行数据重构的步骤，则目标重构网络即为目标编码网络。初始生成网络，是指生成缺失点云数据并利用其生成训练修复点云数据的网络。
相关技术中，模型训练时采用的训练数据中数据部分和标签部分一一对应，在这种情况下，模型仅能从数据部分的总体情况的角度学习训练数据的全局结构，并基于该全局结构进行特征提取。该全局结构的获取情况依赖于 数据部分相较于标签部分的缺失程度，因此通常准确度不足。为了得到更优的全局结构，进而得到能够更加准确地反映训练缺失点云数据的输入特征。
具体的，本申请中，目标编码网络分别以某些训练缺失点云数据为锚点点云进行比对学习。锚点点云，是指作为比对学习的学习基准的点云，与锚点点云对应于同一原始点云数据的训练缺失点云数据为正样本，对应于其他的原始点云数据的训练缺失点云数据为负样本。在训练过程中，训练缺失点云数据输入目标编码网络后，可以得到对应的输入特征。输入特征被输入初始生成网络后得到缺失点云数据，缺失点云数据用于生成训练修复点云数据。
在一种实施方式中，为了提高初始模型的收敛速度，可以利用经过预训练的目标重构网络构成初始模型，预训练会使得目标重构网络的参数基本确定，在初始模型训练时，仅需在已有基础上对其进行微调，同时调节初始生成网络的参数即可。具体的，初始模型的生成过程，包括：
步骤21：利用训练缺失点云数据对初始重构网络进行比对学习训练，得到目标重构网络。
步骤22：利用目标重构网络与初始生成网络组合得到初始模型。
初始重构网络，是指未经过训练的重构网络，利用缺失点云数据，可以对其进行预训练，预训练同样为比对学习训练，得到目标重构网络。通过将目标重构网络与初始生成网络进行组合，即可得到初始模型。在后续对初始模型训练时，由于目标重构网络经过预训练，基本达到收敛，因此相比利用完全没有经过训练的初始重构网络作为目标重构网络并组成初始模型的方案，具有经过预训练的目标初始模型能够更快达到收敛。
具体的，利用训练缺失点云数据对初始重构网络进行比对学习训练，得到目标重构网络的过程可以包括如下步骤：
步骤31：从训练缺失点云数据中确定锚点点云。
步骤32：基于锚点点云，将训练缺失点云数据输入初始重构网络，得到目标数据。
步骤33：利用输入特征得到对比学习损失值，利用重构点云数据得到重建损失值，并利用对比学习损失值和重建损失值对初始重构网络进行参数调整。
步骤34：若检测到满足预训练完成条件，则确定初始重构网络为目标重构网络。
在本实施例中，初始重构网络不仅对输入的训练缺失点云数据进行特征提取，还用于基于提取到的特征进行数据重构，以便将训练缺失点云数据中的噪声去除。因此，目标数据包括输入特征和重构点云数据。输入特征，是指输入的训练缺失点云数据经过特征提取后得到的特征；重构点云数据，是指利用输入特征经过数据重构后得到的重构数据。
在本实施例中，可以利用P in表示原始点云数据，利用S in表示原始点云数据组成的集合，利用S S表示训练缺失点云数据组成的集合。在训练初始重构网络时，可以选择任意一个训练缺失点云数据作为锚点点云P S，并将S S中与P S相对应的训练缺失点云数据作为正样本，将其他的训练缺失点云数据作为负样本。例如，选中作为锚点点云对应的原始点云数据为飞机点云，则S S中与飞机相对应的训练缺失点云数据(即根据飞机点云得到的各个具有缺失的飞机点云)为正样本，与飞机不对应的训练缺失点云数据(例如具有缺失的椅子点云)为负样本。可以理解的是，在将正样本或负样本输入初始重构网络中时，需要声明对应的样本类型(即正样本或负样本)。
在得到目标数据后，分别利用输入特征和重构点云数据计算对应的损失值。具体的，利用输入特征得到对比学习损失值，对比学习损失值，是指用于对特征提取部分进行参数调整的损失值；利用重构点云数据计算得到重建损失值，重建损失值，是指用于对数据重建部分进行参数调整的损失值。在得到上述两种损失值后，利用其对初始重构网络进行参数调整，并在满足预训练完成条件时将初始重构网络确定为目标重构网络。其中，预训练完成条件的具体内容和形式不做限定，例如可以为训练轮次条件，或者可以为训练时间条件，或者可以为其他任意可选的条件。
具体的，将训练缺失点云数据输入初始重构网络，得到目标数据的过程可以包括如下步骤：
步骤41：将训练缺失点云数据输入初始重构网络中的初始编码网络，得到输入特征。
步骤42：将输入特征输入初始重构网络中的初始解码网络，得到重构点云数据。
相应的，利用对比学习损失值和重建损失值对初始重构网络进行参数调整的过程可以包括如下步骤：
步骤43：利用所述对比学习损失值和所述重建损失值生成第一损失值。
步骤44：利用所述第一损失值对所述初始重构网络进行参数调整。
在本实施例中，初始重构网络包括初始编码网络和初始解码网络，目标编码网络用于对训练缺失点云数据进行特征提取，得到输入特征。而初始解码网络用于对输入特征进行解码，以便完成数据重构，得到重构点云数据。将对比学习损失值和重建损失值进行整合后，可以得到第一损失值，进而利用第一损失值对整个初始重构网络进行参数调整。对于第一损失值的具体生成方式，本实施例不做限定，例如在一种实施方式中，可以将二者相加得到第一损失值。
在一种实施方式中，初始编码网络，可以采用PointNet++(一种处理点云的网络结构)作为基本框架，其中包括若干个特征提取块，每个特征提取块包含一个MLP(Multi-layer Perceptron，多层感知机)和一个下采样层。MLP用于优化提取的点云特征，利用基于FPS(Farthest Point Sampling，最远点采样)的下采样层对点云进行下采样，由细到粗得到多个分辨率的点云，从而学习多个尺度的点云局部特征，最后利用初始编码网络中的池化层进行池化处理，得到点云的全局特征。初始解码网络，包括多个MLP进行特征维度变换并利用多个上采样层进行上采样，其能够迭代地对输入点云的形状进行重构。该结构的初始编码网络和初始解码网络配合，可以更好地去除输入点云中噪声，优化输入点云形状。
可以理解的是，由于对于一个原始点云数据，存在多个不同缺失情况的训练缺失点云数据，且各个训练缺失点云数据具有相同的全局结构。然而，不同的训练缺失点云数据作为同一个原始点云数据的不同局部部分，都有一个有限的感受野，利用对比学习训练初始编码网络，能够使得初始编码网络学习到的点云全局结构包含来自不同局部区域的信息。
举例说明上述过程：将类别为“飞机”的训练缺失点云数据输入初始重构网络，初始编码网络可以得到表示飞机的每个部分的局部细节特征，以及表示整体的全局结构特征，即全局结构。以同样的方式获取对比学习正负样本的全局结构，作为初始解码网络的输入，得到重构点云数据。然后，计算 最小化对比学习损失和重构损失，利用其更新网络参数，进而不断优化提取到输入点云的局部和全局特征。具体的，可以利用L i NCE表示对比学习损失值，利用L in表示重构损失值，采用InfoNCE loss作为对比学习损失值的损失函数，具体计算公式为：
Figure PCTCN2022078359-appb-000001
其中，v表示锚点点云的特征，v+表示正样本的输入特征，v-表示负样本的输入特征，
Figure PCTCN2022078359-appb-000002
表示所有正样本的输入特征组成的集合，
Figure PCTCN2022078359-appb-000003
表示所有负样本的输入特征组成的集合，τ为常数。
同时，可以利用：
Figure PCTCN2022078359-appb-000004
计算重构损失，其中S 1为重构点云数据，S 2为训练缺失点云数据对应的原始点云数据，x和y表示其中的点。
基于上述实施例，在一种可行的实施方式中，初始生成网络可以根据生成的缺失点云数据和训练缺失点云数据(或经过重构得到的重构点云数据)直接拼接得到训练修复点云数据。在另一种实施方式中，可以将直接拼接得到的数据为粗糙的三维点云数据，初始生成网络可以对粗糙三维点云数据进行进一步优化，得到训练修复点云数据。具体的，初始生成网络包括缺失点云生成网络和修正网络，训练修复点云数据的生成过程可以包括如下步骤：
步骤51：将输入特征输入缺失点云生成网络得到缺失点云数据。
步骤52：将缺失点云数据和目标重构网络输出的输出数据输入修正网络，得到训练修复点云数据。
其中，缺失点云生成网络，是指用于根据输入特征生成对应的缺失点云数据的网络。修正网络，是指对输出数据(可以为未经重构的训练缺失点云数据，或者为经过重构的重构点云数据)进行形状修正的网络。缺失点云生成网络和修正网络的具体结构不做限定，可以根据需要进行设置。
例如在一种实施方式中，缺失点云生成网络包括缺失点云调制模块和折叠解码模块，将输入特征输入缺失点云生成网络得到缺失点云数据的过程可以包括如下步骤：
步骤53：将输入特征输入缺失点云调制模块，得到缺失点云特征。
步骤54：将缺失点云特征、输入特征输入折叠解码模块，得到缺失点云数据。
具体的，缺失点云生成网络包括多个解码模块，每个解码模块包含一个缺失点云调制模块和一个基于折叠的解码层(即折叠解码模块)。缺失点云调制模块，通过一个MLP将输入特征进行变换，作为学习得到的缺失点云特征。基于折叠的解码层，对随机采样得到的二维网格、学习得到的缺失点云特征和输入特征进行处理，得到缺失点云数据。通过逐层提高二维网格的密度的方式，可以预测更高分辨率的缺失点云。
本实施例并不限定修正网络得到训练修复点云数据的具体过程，在一种实施方式中，修正网络融合重构点云数据和缺失点云数据后，利用FPS采样得到粗糙三维点云。修正网络包括多个MLP和一个基于折叠的修正层，对于输入的粗糙三维点云，经过多个MLP处理，可以得到点云特征，然后从固定尺寸的二维平面中随机采样二维网格。将采样得到的二维网格，点云特征、点云的三维坐标输入基于折叠的修正层，利用其优化粗糙三维点云，得到训练修复点云数据。
可以理解的是，在存在修正网络的情况下，基于训练修复点云数据和训练缺失点云数据调整初始模型的参数的过程可以包括如下步骤：
步骤61：利用训练修复点云数据和原始点云数据得到修正重建损失值。
步骤62：利用缺失点云数据与缺失点云真值数据得到缺失重建损失值。
步骤63：利用修正重建损失值和缺失重建损失值生成第二损失值。
步骤64：利用第二损失值对初始模型进行参数调整其中，缺失真值数据为训练缺失点云数据与对应的原始点云数据的差值数据。可以利用L r表示修正重建损失值，利用L c表示缺失重建损失值。L r和L c的计算方式与L in相同。
请参考图2，图2为本申请实施例提供的一种具体的点云补全模型结构图。其中，不完整的三维点云即为训练缺失点云数据或模型训练好被使用时输入的待补全点云数据，基于对比学习的输入点云重建网络即为目标重构网络， 缺失点云解码调制网络即为缺失点云生成网络，粗糙点云预测修正网络即为修正网络。其中，模块1即为目标编码网络(或初始编码网络)，用于特征编码，模块2即为初始解码网络，用于全连接解码，模块3即为折叠解码模块，用于折叠解码，模块4即为修正网络，用于进行粗糙点云修正，模块5即为缺失点云调制模块，用于进行缺失点云调制，生成缺失点云特征。图2中的各个损失值计算方式可以参考上述过程，在此不再赘述。
可以理解的是，在模型训练完毕后，则可以利用其进行点云数据补全，因此本申请还提供了一种点云缺失补全方法，该方法可以包括如下步骤：
步骤71：获取待补全点云数据。
步骤72：将待补全点云数据输入如上述的点云补全模型，得到处理后点云数据。
应用本申请实施例提供的模型训练方法，初始模型中包括目标重构网络和初始生成网络，其中，目标重构网络能够以原始点云数据为锚点，从具有不同缺失情况的训练缺失点云数据的角度对全局结构进行学习。即对应于同一个原始点云数据的若干个训练缺失点云数据，其具有相同的全局结构，但是由于缺失的部分不同，使得其具有有限且不同的感受野，基于比对学习的训练方式，使得网络学习到的点云全局结构能够包含来自不同局部区域的信息，进而能够进行更准确地特征提取。初始生成网络用于生成缺失点云数据，其基于训练缺失点云数据对应的输入特征对训练缺失点云数据失去的缺失点云部分进行推测。在训练时，根据输入特征学习提取缺失点云特征。在初始模型满足训练完成条件时，将其确定为点云补全模型。点云补全模型能够获取具有局部区域信息的全局结构，并根据输入数据的情况对缺失点云进行准确地预测，进而提高了补全处理后的处理后点云数据的准确性，解决了相关技术存在的数据准确性低的问题。
下面对本申请实施例提供的模型训练装置进行介绍，下文描述的模型训练装置与上文描述的模型训练方法可相互对应参照。
请参考图3，图3为本申请实施例提供的一种模型训练装置的结构示意图，包括：
第一获取模块110，用于获取训练缺失点云数据；
训练模块120，用于将训练缺失点云数据输入初始模型，得到训练修复点云数据，并基于训练修复点云数据和训练缺失点云数据对应的原始点云数据调整初始模型的参数；
确定模块130，用于若检测到满足训练完成条件，则确定初始模型为点云补全模型；
其中，初始模型包括目标重构网络和初始生成网络，目标重构网络包括目标编码网络，目标编码网络利用训练缺失点云数据进行对比学习，训练缺失点云数据输入目标编码网络得到输入特征，输入特征输入初始生成网络得到缺失点云数据，缺失点云数据用于生成训练修复点云数据。
可选地，包括：
预训练模块，用于利用训练缺失点云数据对初始重构网络进行学习训练，得到目标重构网络；
组合模块，用于利用目标重构网络与初始生成网络组合得到初始模型。
可选地，预训练模块，包括：
锚点确定单元，用于从训练缺失点云数据中确定锚点点云；
输入单元，用于基于锚点点云，将训练缺失点云数据输入初始重构网络，得到目标数据；其中，目标数据包括输入特征和重构点云数据；
参数调节单元，用于利用输入特征得到对比学习损失值，利用重构点云数据得到重建损失值，并利用对比学习损失值和重建损失值对初始重构网络进行参数调整；
目标重构网络确定单元，用于若检测到满足预训练完成条件，则确定初始重构网络为目标重构网络。
可选地，输入单元，包括：
特征获取子单元，用于将训练缺失点云数据输入初始重构网络中的初始编码网络，得到输入特征；
重构子单元，用于将输入特征输入初始重构网络中的初始解码网络，得到重构点云数据；
相应的，参数调节单元，包括：
第一损失生成子单元，用于利用对比学习损失值和重建损失值生成第一损失值；
初始重构网络调节子单元，用于利用第一损失值对初始重构网络进行参数调整。
可选地，初始编码网络包括若干个特征提取块，每个特征提取块包括一个多层感知机和一个基于最远点采样的下采样层；初始解码网络包括多个多层感知机和多个上采样层。
可选地，第一获取模块110，包括：
原始缺失获取单元，用于获取若干个原始缺失点云作为原始点云数据；
缺失处理单元，用于分别对各个原始缺失点云进行不同程度的缺失处理，得到训练缺失点云数据；缺失处理为裁剪处理。
可选地，训练模块120，包括：
缺失点云生成单元，将输入特征输入缺失点云生成网络得到缺失点云数据；
修正单元，用于将缺失点云数据和目标重构网络输出的输出数据输入修正网络，得到训练修复点云数据；
其中，缺失点云生成网络包括缺失点云调制模块和折叠解码模块，缺失点云生成单元，包括：
缺失特征获取子单元，用于将输入特征输入缺失点云调制模块，得到缺失点云特征；
折叠解码子单元，用于将缺失点云特征、输入特征输入折叠解码模块，得到缺失点云数据。
可选地，训练模块120，包括：
修正重建损失生成单元，用于利用训练修复点云数据和原始点云数据得到修正重建损失值；
缺失重建损失生成单元，用于利用缺失点云数据与缺失点云真值数据得到缺失重建损失值；
第二损失生成单元，用于利用修正重建损失值和缺失重建损失值生成第二损失值；
初始模型调节单元，用于利用第二损失值对初始模型进行参数调整。
下面对本申请实施例提供的点云缺失补全装置进行介绍，下文描述的点云缺失补全装置与上文描述的点云缺失补全方法可相互对应参照。
请参考图4，图4为本申请实施例提供的一种点云缺失补全装置的结构示意图，包括：
第二获取模块210，用于获取待补全点云数据；
补全处理模块220，用于将所述待补全点云数据输入上述的点云补全模型，得到处理后点云数据。
下面对本申请实施例提供的电子设备进行介绍，下文描述的电子设备与上文描述的模型训练方法可相互对应参照。
请参考图5，图5为本申请实施例提供的一种电子设备的结构示意图。其中电子设备100可以包括处理器101和存储器102，还可以进一步包括多媒体组件103、信息输入/信息输出(I/O)接口104以及通信组件105中的一种或多种。
其中，处理器101用于控制电子设备100的整体操作，以完成上述的模型训练方法中的全部或部分步骤；存储器102用于存储各种类型的数据以支持在电子设备100的操作，这些数据例如可以包括用于在该电子设备100上操作的任何应用程序或方法的指令，以及应用程序相关的数据。该存储器102可以由任何类型的易失性或非易失性存储设备或者它们的组合实现，例如静态随机存取存储器(Static Random Access Memory，SRAM)、电可擦除可编程只读存储器(Electrically Erasable Programmable Read-Only Memory，EEPROM)、可擦除可编程只读存储器(Erasable Programmable Read-Only Memory，EPROM)、可编程只读存储器(Programmable Read-Only Memory，PROM)、只读存储器(Read-Only Memory，ROM)、磁存储器、快闪存储器、磁盘或光盘中的一种或多种。
多媒体组件103可以包括屏幕和音频组件。其中屏幕例如可以是触摸屏，音频组件用于输出和/或输入音频信号。例如，音频组件可以包括一个麦克风，麦克风用于接收外部音频信号。所接收的音频信号可以被进一步存储在存储器102或通过通信组件105发送。音频组件还包括至少一个扬声器，用于输出音频信号。I/O接口104为处理器101和其他接口模块之间提供接口，上述其他接口模块可以是键盘，鼠标，按钮等。这些按钮可以是虚拟按钮或者实体按 钮。通信组件105用于电子设备100与其他设备之间进行有线或无线通信。无线通信，例如Wi-Fi，蓝牙，近场通信(Near Field Communication，简称NFC)，2G、3G或4G，或它们中的一种或几种的组合，因此相应的该通信组件105可以包括：Wi-Fi部件，蓝牙部件，NFC部件。
电子设备100可以被一个或多个应用专用集成电路(Application Specific Integrated Circuit，简称ASIC)、数字信号处理器(Digital Signal Processor，简称DSP)、数字信号处理设备(Digital Signal Processing Device，简称DSPD)、可编程逻辑器件(Programmable Logic Device，简称PLD)、现场可编程门阵列(Field Programmable Gate Array，简称FPGA)、控制器、微控制器、微处理器或其他电子元件实现，用于执行上述实施例给出的模型训练方法。
下面对本申请实施例提供的计算机可读存储介质进行介绍，下文描述的计算机可读存储介质与上文描述的模型训练方法可相互对应参照。
本申请还提供一种计算机可读存储介质，计算机可读存储介质上存储有计算机程序，计算机程序被处理器执行时实现上述的模型训练方法的步骤。
该计算机可读存储介质可以包括：U盘、移动硬盘、只读存储器(Read-Only Memory，ROM)、随机存取存储器(Random Access Memory，RAM)、磁碟或者光盘等各种可以存储程序代码的介质。
本说明书中各个实施例采用递进的方式描述，每个实施例重点说明的都是与其它实施例的不同之处，各个实施例之间相同或相似部分互相参见即可。对于实施例公开的装置而言，由于其与实施例公开的方法相对应，所以描述的比较简单，相关之处参见方法部分说明即可。
本领域技术人员还可以进一步意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，能够以电子硬件、计算机软件或者二者的结合来实现，为了清楚地说明硬件和软件的可互换性，在上述说明中已经按照功能一般性地描述了各示例的组成及步骤。这些功能究竟以硬件还是软件的方式来执行，取决于技术方案的特定应用和设计约束条件。本领域技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应该认为超出本申请的范围。
结合本文中所公开的实施例描述的方法或算法的步骤可以直接用硬件、处理器执行的软件模块，或者二者的结合来实施。软件模块可以置于随机存储器(RAM)、内存、只读存储器(ROM)、电可编程ROM、电可擦除可编程ROM、寄存器、硬盘、可移动磁盘、CD-ROM、或技术领域内所公知的任意其它形式的存储介质中。
最后，还需要说明的是，在本文中，诸如第一和第二等之类的关系属于仅仅用来将一个实体或者操作与另一个实体或者操作区分开来，而不一定要求或者暗示这些实体或操作之间存在任何这种实际的关系或者顺序。而且，术语包括、包含或者其他任何变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者设备不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者设备所固有的要素。
本文中应用了具体个例对本申请的原理及实施方式进行了阐述，以上实施例的说明只是用于帮助理解本申请的方法及其核心思想；同时，对于本领域的一般技术人员，依据本申请的思想，在具体实施方式及应用范围上均会有改变之处，综上所述，本说明书内容不应理解为对本申请的限制。

Claims (13)

  1. 一种模型训练方法，其特征在于，包括：
    获取训练缺失点云数据；
    将所述训练缺失点云数据输入初始模型，得到训练修复点云数据，并基于所述训练修复点云数据和所述训练缺失点云数据对应的原始点云数据调整所述初始模型的参数；
    若检测到满足训练完成条件，则确定所述初始模型为点云补全模型；
    其中，所述初始模型包括目标重构网络和初始生成网络，所述目标重构网络包括目标编码网络，所述目标编码网络利用所述训练缺失点云数据进行对比学习，所述训练缺失点云数据输入所述目标编码网络得到输入特征，所述输入特征输入所述初始生成网络得到缺失点云数据，所述缺失点云数据用于生成所述训练修复点云数据。
  2. 根据权利要求1所述的模型训练方法，其特征在于，所述初始模型的生成过程，包括：
    利用所述训练缺失点云数据对初始重构网络进行学习训练，得到所述目标重构网络；
    利用所述目标重构网络与所述初始生成网络组合得到所述初始模型。
  3. 根据权利要求2所述的模型训练方法，其特征在于，所述利用所述训练缺失点云数据对初始重构网络进行学习训练，得到所述目标重构网络，包括：
    从所述训练缺失点云数据中确定锚点点云；
    基于所述锚点点云，将所述训练缺失点云数据输入所述初始重构网络，得到目标数据；其中，所述目标数据包括所述输入特征和重构点云数据；
    利用所述输入特征得到对比学习损失值，利用所述重构点云数据得到重建损失值，并利用所述对比学习损失值和所述重建损失值对所述初始重构网络进行参数调整；
    若检测到满足预训练完成条件，则确定所述初始重构网络为所述目标重构网络。
  4. 根据权利要求3所述的模型训练方法，其特征在于，所述将所述训练缺失点云数据输入所述初始重构网络，得到目标数据，包括：
    将所述训练缺失点云数据输入所述初始重构网络中的初始编码网络，得到所述输入特征；
    将所述输入特征输入所述初始重构网络中的初始解码网络，得到所述重构点云数据；
    相应的，所述利用所述对比学习损失值和所述重建损失值对所述初始重构网络进行参数调整，包括：
    利用所述对比学习损失值和所述重建损失值生成第一损失值；
    利用所述第一损失值对所述初始重构网络进行参数调整。
  5. 根据权利要求4所述的模型训练方法，其特征在于，所述初始编码网络包括若干个特征提取块，每个所述特征提取块包括一个多层感知机和一个基于最远点采样的下采样层；所述初始解码网络包括多个多层感知机和多个上采样层。
  6. 根据权利要求1所述的模型训练方法，其特征在于，所述获取训练缺失点云数据，包括：
    获取若干个原始缺失点云作为所述原始点云数据；
    分别对各个所述原始缺失点云进行不同程度的缺失处理，得到所述训练缺失点云数据；所述缺失处理为裁剪处理。
  7. 根据权利要求1所述的模型训练方法，其特征在于，所述初始生成网络包括缺失点云生成网络和修正网络，所述训练修复点云数据的生成过程包括：
    将所述输入特征输入所述缺失点云生成网络得到所述缺失点云数据；
    将所述缺失点云数据和所述目标重构网络输出的输出数据输入所述修正网络，得到所述训练修复点云数据；
    其中，所述缺失点云生成网络包括缺失点云调制模块和折叠解码模块，所述将所述输入特征输入所述缺失点云生成网络得到所述缺失点云数据，包括：
    将所述输入特征输入所述缺失点云调制模块，得到缺失点云特征；
    将所述缺失点云特征、所述输入特征输入所述折叠解码模块，得到所述缺失点云数据。
  8. 根据权利要求7所述的模型训练方法，其特征在于，所述基于所述训练修复点云数据和所述训练缺失点云数据对应的原始点云数据调整所述初始模型的参数，包括：
    利用所述训练修复点云数据和原始点云数据得到修正重建损失值；
    利用所述缺失点云数据与缺失点云真值数据得到缺失重建损失值；
    利用所述修正重建损失值和所述缺失重建损失值生成第二损失值；
    利用所述第二损失值对所述初始模型进行参数调整；
    其中，所述缺失点云真值数据为所述训练缺失点云数据与对应的原始点云数据的差值数据。
  9. 一种点云缺失补全方法，其特征在于，包括：
    获取待补全点云数据；
    将所述待补全点云数据输入如权利要求1至8任一项所述的点云补全模型，得到处理后点云数据。
  10. 一种模型训练装置，其特征在于，包括：
    第一获取模块，用于获取训练缺失点云数据；
    训练模块，用于将所述训练缺失点云数据输入初始模型，得到训练修复点云数据，并基于所述训练修复点云数据和所述训练缺失点云数据对应的原始点云数据调整所述初始模型的参数；
    确定模块，用于若检测到满足训练完成条件，则确定所述初始模型为点云补全模型；
    其中，所述初始模型包括目标重构网络和初始生成网络，所述目标重构网络包括目标编码网络，所述目标编码网络利用所述训练缺失点云数据进行对比学习，所述训练缺失点云数据输入所述目标编码网络得到输入特征，所述输入特征输入所述初始生成网络得到缺失点云数据，所述缺失点云数据用于生成所述训练修复点云数据。
  11. 一种点云缺失补全装置，其特征在于，包括：
    第二获取模块，用于获取待补全点云数据；
    补全处理模块，用于将所述待补全点云数据输入如权利要求1至8任一项所述的点云补全模型，得到处理后点云数据。
  12. 一种电子设备，其特征在于，包括存储器和处理器，其中：
    所述存储器，用于保存计算机程序；
    所述处理器，用于执行所述计算机程序，以实现如权利要求1至8任一项所述的模型训练方法，和/或，如权利要求9所述的点云缺失补全方法。
  13. 一种计算机可读存储介质，其特征在于，用于保存计算机程序，其中，所述计算机程序被处理器执行时实现如权利要求1至8任一项所述的模型训练方法，和/或，如权利要求9所述的点云缺失补全方法。
PCT/CN2022/078359 2021-09-26 2022-02-28 模型训练、点云缺失补全方法、装置、设备及介质 WO2023045252A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202111129999.6A CN113850916A (zh) 2021-09-26 2021-09-26 模型训练、点云缺失补全方法、装置、设备及介质
CN202111129999.6 2021-09-26

Publications (1)

Publication Number Publication Date
WO2023045252A1 true WO2023045252A1 (zh) 2023-03-30

Family

ID=78979820

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/078359 WO2023045252A1 (zh) 2021-09-26 2022-02-28 模型训练、点云缺失补全方法、装置、设备及介质

Country Status (2)

Country Link
CN (1) CN113850916A (zh)
WO (1) WO2023045252A1 (zh)

Cited By (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116524123A (zh) * 2023-04-20 2023-08-01 深圳市元甪科技有限公司 一种三维电阻抗断层扫描图像重建方法及相关设备
CN116721399A (zh) * 2023-07-26 2023-09-08 之江实验室 一种量化感知训练的点云目标检测方法及装置
CN116882035A (zh) * 2023-09-07 2023-10-13 湖南省国土资源规划院 一种基于人工智能的空间对象识别与建模方法及相关设备
CN116910912A (zh) * 2023-07-28 2023-10-20 小米汽车科技有限公司 车辆三维模型的生成方法、装置、设备及存储介质
CN116957991A (zh) * 2023-09-19 2023-10-27 北京渲光科技有限公司 三维模型补全方法以及三维模型补全模型的生成方法
CN117115366A (zh) * 2023-10-25 2023-11-24 中国科学院自动化研究所 基于无人系统三维感知的环境模型重建方法、系统及设备
CN117173069A (zh) * 2023-11-01 2023-12-05 湖北工业大学 一种边坡结构面点云数据补全方法
CN117671131A (zh) * 2023-10-20 2024-03-08 南京邮电大学 一种基于深度学习的工业零件三维点云修复方法及装置
CN117807875A (zh) * 2023-12-28 2024-04-02 上海强华实业股份有限公司 石英器件的三维数据逆向重建和尺寸测量系统及方法
CN117975202A (zh) * 2024-04-01 2024-05-03 之江实验室 模型训练方法、业务执行方法、装置、介质及设备
CN118038085A (zh) * 2024-04-09 2024-05-14 无锡学院 一种基于孪生网络的点云关键点检测方法及装置
CN118211132A (zh) * 2024-05-21 2024-06-18 之江实验室 一种基于点云的三维人体表面数据生成方法及装置

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113850916A (zh) * 2021-09-26 2021-12-28 浪潮电子信息产业股份有限公司 模型训练、点云缺失补全方法、装置、设备及介质
CN114331821B (zh) * 2021-12-29 2023-09-22 中国人民解放军火箭军工程大学 一种图像转换方法及系统
CN114820465B (zh) * 2022-04-06 2024-04-26 合众新能源汽车股份有限公司 点云检测模型训练方法、装置、电子设备及存储介质
CN114758078B (zh) * 2022-05-17 2024-03-15 北京大学深圳研究生院 一种点云数据处理方法、装置、电子设备及存储介质
CN114842180B (zh) * 2022-07-06 2022-12-02 山东海量信息技术研究院 一种点云补全方法、装置、设备及介质
CN115422264B (zh) * 2022-11-02 2023-05-05 苏州浪潮智能科技有限公司 一种时序数据处理方法、装置、设备及可读存储介质
CN115994936B (zh) * 2023-03-23 2023-06-30 季华实验室 点云融合模型获取方法、装置、电子设备及存储介质
CN116758096A (zh) * 2023-07-03 2023-09-15 强联智创（北京）科技有限公司 动脉瘤分割方法、电子设备及存储介质
CN117422847B (zh) * 2023-10-27 2024-08-20 神力视界(深圳)文化科技有限公司 模型修复方法、装置、电子设备和计算机存储介质
CN117807378A (zh) * 2023-12-01 2024-04-02 太极计算机股份有限公司 一种风电数据智能修复方法、装置

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190325604A1 (en) * 2018-04-20 2019-10-24 Streem, Inc. Augmentation of 3-d point clouds with subsequently captured data
CN111582105A (zh) * 2020-04-28 2020-08-25 清华大学 基于局部全局双向推理的无监督点云特征学习方法及装置
CN113205104A (zh) * 2021-04-23 2021-08-03 广西大学 一种基于深度学习的点云补全方法
CN113850916A (zh) * 2021-09-26 2021-12-28 浪潮电子信息产业股份有限公司 模型训练、点云缺失补全方法、装置、设备及介质

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190325604A1 (en) * 2018-04-20 2019-10-24 Streem, Inc. Augmentation of 3-d point clouds with subsequently captured data
CN111582105A (zh) * 2020-04-28 2020-08-25 清华大学 基于局部全局双向推理的无监督点云特征学习方法及装置
CN113205104A (zh) * 2021-04-23 2021-08-03 广西大学 一种基于深度学习的点云补全方法
CN113850916A (zh) * 2021-09-26 2021-12-28 浪潮电子信息产业股份有限公司 模型训练、点云缺失补全方法、装置、设备及介质

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
ANTONIO ALLIEGRO; DIEGO VALSESIA; GIULIA FRACASTORO; ENRICO MAGLI; TATIANA TOMMASI: "Denoise and Contrast for Category Agnostic Shape Completion", ARXIV.ORG, CORNELL UNIVERSITY LIBRARY, 201 OLIN LIBRARY CORNELL UNIVERSITY ITHACA, NY 14853, 30 March 2021 (2021-03-30), 201 Olin Library Cornell University Ithaca, NY 14853 , XP081919641 *
NAZIR DANISH, AFZAL MUHAMMAD ZESHAN, PAGANI ALAIN, LIWICKI MARCUS, STRICKER DIDIER: "Contrastive Learning for 3D Point Clouds Classification and Shape Completion", SENSORS, vol. 21, no. 21, pages 7392, XP093053988, DOI: 10.3390/s21217392 *

Cited By (20)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116524123B (zh) * 2023-04-20 2024-02-13 深圳市元甪科技有限公司 一种三维电阻抗断层扫描图像重建方法及相关设备
CN116524123A (zh) * 2023-04-20 2023-08-01 深圳市元甪科技有限公司 一种三维电阻抗断层扫描图像重建方法及相关设备
CN116721399A (zh) * 2023-07-26 2023-09-08 之江实验室 一种量化感知训练的点云目标检测方法及装置
CN116721399B (zh) * 2023-07-26 2023-11-14 之江实验室 一种量化感知训练的点云目标检测方法及装置
CN116910912A (zh) * 2023-07-28 2023-10-20 小米汽车科技有限公司 车辆三维模型的生成方法、装置、设备及存储介质
CN116910912B (zh) * 2023-07-28 2024-04-30 小米汽车科技有限公司 车辆三维模型的生成方法、装置、设备及存储介质
CN116882035A (zh) * 2023-09-07 2023-10-13 湖南省国土资源规划院 一种基于人工智能的空间对象识别与建模方法及相关设备
CN116882035B (zh) * 2023-09-07 2023-11-21 湖南省国土资源规划院 一种基于人工智能的空间对象识别与建模方法及相关设备
CN116957991A (zh) * 2023-09-19 2023-10-27 北京渲光科技有限公司 三维模型补全方法以及三维模型补全模型的生成方法
CN116957991B (zh) * 2023-09-19 2023-12-15 北京渲光科技有限公司 三维模型补全方法
CN117671131A (zh) * 2023-10-20 2024-03-08 南京邮电大学 一种基于深度学习的工业零件三维点云修复方法及装置
CN117115366B (zh) * 2023-10-25 2024-02-13 中国科学院自动化研究所 基于无人系统三维感知的环境模型重建方法、系统及设备
CN117115366A (zh) * 2023-10-25 2023-11-24 中国科学院自动化研究所 基于无人系统三维感知的环境模型重建方法、系统及设备
CN117173069A (zh) * 2023-11-01 2023-12-05 湖北工业大学 一种边坡结构面点云数据补全方法
CN117807875A (zh) * 2023-12-28 2024-04-02 上海强华实业股份有限公司 石英器件的三维数据逆向重建和尺寸测量系统及方法
CN117807875B (zh) * 2023-12-28 2024-05-28 上海强华实业股份有限公司 石英器件的三维数据逆向重建和尺寸测量系统及方法
CN117975202A (zh) * 2024-04-01 2024-05-03 之江实验室 模型训练方法、业务执行方法、装置、介质及设备
CN118038085A (zh) * 2024-04-09 2024-05-14 无锡学院 一种基于孪生网络的点云关键点检测方法及装置
CN118038085B (zh) * 2024-04-09 2024-06-07 无锡学院 一种基于孪生网络的点云关键点检测方法及装置
CN118211132A (zh) * 2024-05-21 2024-06-18 之江实验室 一种基于点云的三维人体表面数据生成方法及装置

Also Published As

Publication number Publication date
CN113850916A (zh) 2021-12-28

Similar Documents

Publication Publication Date Title
WO2023045252A1 (zh) 模型训练、点云缺失补全方法、装置、设备及介质
US11798132B2 (en) Image inpainting method and apparatus, computer device, and storage medium
CN110796619B (zh) 一种图像处理模型训练方法、装置、电子设备及存储介质
JP7373554B2 (ja) クロスドメイン画像変換
CN110837811B (zh) 语义分割网络结构的生成方法、装置、设备及存储介质
CN111105017B (zh) 神经网络量化方法、装置及电子设备
CN112927357A (zh) 一种基于动态图网络的3d物体重建方法
KR20190039459A (ko) Fun을 사용하여 cnn의 성능을 향상시키기 위한 학습 방법 및 학습 장치, 그리고 이를 이용한, 테스트 방법 및 테스트 장치
CN112348870B (zh) 一种基于残差融合的显著性目标检测方法
CN115578515B (zh) 三维重建模型的训练方法、三维场景渲染方法及装置
CN117274491B (zh) 三维重建模型的训练方法、装置、设备和介质
RU2745010C1 (ru) Способы реконструкции карты глубины и электронное вычислительное устройство для их реализации
CN116645598A (zh) 一种基于通道注意力特征融合的遥感图像语义分割方法
CN117173412A (zh) 一种基于CNN和Transformer融合网络的医学图像分割方法
CN110633706B (zh) 一种基于金字塔网络的语义分割方法
CN113554047A (zh) 图像处理模型的训练方法、图像处理方法及对应的装置
CN114926734A (zh) 基于特征聚合和注意融合的固体废弃物检测装置及方法
CN114066899A (zh) 图像分割模型训练、图像分割方法、装置、设备及介质
CN113393385B (zh) 基于多尺度融合的无监督去雨方法、系统、装置及介质
CN112561050A (zh) 一种神经网络模型训练方法及装置
WO2021037174A1 (zh) 一种神经网络模型训练方法及装置
CN110570375A (zh) 一种图像处理方法、装置、电子设置以及存储介质
CN117689772A (zh) 一种基于潜空间特征融合的物体背景合成方法和系统
KR101678453B1 (ko) 이미지 처리 장치 및 방법
CN113628140B (zh) 一种基于深度学习的点云稠密补全方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22871318

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE