CN114764746A - Super-resolution method and device for laser radar, electronic device and storage medium - Google Patents

Super-resolution method and device for laser radar, electronic device and storage medium Download PDF

Info

Publication number
CN114764746A
CN114764746A CN202111109236.5A CN202111109236A CN114764746A CN 114764746 A CN114764746 A CN 114764746A CN 202111109236 A CN202111109236 A CN 202111109236A CN 114764746 A CN114764746 A CN 114764746A
Authority
CN
China
Prior art keywords
point cloud
super
resolution
point
distance function
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202111109236.5A
Other languages
Chinese (zh)
Other versions
CN114764746B (en
Inventor
李鹏飞
周谷越
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tsinghua University
Original Assignee
Tsinghua University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tsinghua University filed Critical Tsinghua University
Priority to CN202111109236.5A priority Critical patent/CN114764746B/en
Publication of CN114764746A publication Critical patent/CN114764746A/en
Application granted granted Critical
Publication of CN114764746B publication Critical patent/CN114764746B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/088—Non-supervised learning, e.g. competitive learning

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Molecular Biology (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Mathematical Physics (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Probability & Statistics with Applications (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Biology (AREA)
  • Optical Radar Systems And Details Thereof (AREA)
  • Length Measuring Devices By Optical Means (AREA)

Abstract

The invention provides a super-resolution method and a super-resolution device of a laser radar, an electronic device and a storage medium, wherein the method comprises the following steps: inputting the original point cloud into a discriminant model to obtain the shape embedding of a point cloud block corresponding to the original point cloud; the original point cloud is obtained by constraining the point cloud data of the laser radar obtained by scanning; splicing the three-dimensional coordinates of target grid points of a target resolution and corresponding shape embedding values thereof together and inputting the spliced three-dimensional coordinates into a generative model to obtain a symbol distance function value of the target grid points; the shape embedding value corresponding to the target grid point is obtained through linear interpolation based on the shape embedding of the point cloud block; and the symbol distance function value is used for obtaining a super-resolution reconstruction result. The discriminant model and the generative model are obtained by performing collaborative training on the point cloud sample data and the corresponding reconstructed point cloud sample data. The invention can realize the rapid super-resolution reconstruction process of the new laser radar data on any resolution.

Description

Super-resolution method and device for laser radar, electronic equipment and storage medium
Technical Field
The present invention relates to the field of optics, and in particular, to a super-resolution method and apparatus for a laser radar, an electronic device, and a storage medium.
Background
The laser radar is an optical remote sensing technology, and measures parameters such as a distance of a target by emitting pulse laser to the target. Super-resolution is a process of improving the resolution of an original image by a hardware or software method. The symbolic distance function may be expressed as a signed distance value from a point in a metric space to a subset region boundary of the space: the point is positive inside the region boundary, negative outside, and 0 on the boundary.
The prior art of processing laser radar data and super-resolution reconstruction of images independently is more, and research on super-resolution reconstruction technology surrounding the laser radar data is less. In addition, in most of the technologies, super-resolution reconstruction is realized by adopting a method combining multi-frame images or combining 3D laser radar data and 2D image data, a technology for directly performing super-resolution reconstruction on single-frame 3D laser radar data is not available, and the defects of high complexity, low processing speed and high equipment cost exist. More importantly, the prior art super-resolution reconstruction is limited to reconstruction to a specified high resolution and does not achieve the goal of reconstruction to any resolution.
Disclosure of Invention
The invention provides a super-resolution method and device for a laser radar, an electronic device and a storage medium, which are used for solving the technical defects in the prior art.
The invention provides a super-resolution method of a laser radar, which comprises the following steps:
inputting the original point cloud into a discriminant model to obtain the shape embedding of a point cloud block corresponding to the original point cloud; the original point cloud is obtained by constraining point cloud data of the laser radar obtained by scanning;
splicing the three-dimensional coordinates of target grid points of a target resolution and corresponding shape embedded values thereof together and inputting the spliced three-dimensional coordinates into a generative model to obtain a symbol distance function value of the target grid points; the shape embedding value corresponding to the target grid point is obtained through linear interpolation based on the shape embedding of the point cloud block; the symbol distance function value is used for forming a point cloud result after super-resolution reconstruction;
and the discriminant model and the generative model are obtained by performing collaborative training on the point cloud sample data and the corresponding reconstructed point cloud sample data.
The super-resolution method for the laser radar according to the present invention includes, after the three-dimensional coordinates of the target grid points and the corresponding shape embedding values are spliced together and input into the generative model, obtaining a symbol distance function value of the target grid points:
and selecting the grid points for optimizing completion based on the comparison result of the absolute value of the sign distance function value of the target grid points and a preset threshold value so as to form a point cloud result after super-resolution reconstruction.
The super-resolution method for the laser radar according to the present invention is a method for selecting a grid point for optimization completion based on a comparison result between an absolute value of a symbol distance function value of the target grid point and a preset threshold, including:
comparing the absolute value of the symbol distance function value of the target grid point with a preset threshold value to obtain a comparison result;
and if the comparison result is that the absolute value of the symbol distance function value of the target grid point is smaller than a preset threshold value, taking the target grid point as a grid point for optimizing completion.
The super-resolution method of the laser radar according to the present invention, wherein the method further comprises:
inputting the original point cloud into the discriminant model to obtain a point cloud which is completed under the same corresponding resolution and has a category probability value;
and the discriminant model is obtained by training point cloud sample data and the corresponding reconstructed point cloud class sample.
The super-resolution method of the laser radar is characterized in that the discriminant model and the generative model are cooperatively trained loss functions
Figure BDA0003273663140000031
Comprises the following steps:
Figure BDA0003273663140000032
Figure BDA0003273663140000033
Figure BDA0003273663140000034
wherein,
Figure BDA0003273663140000035
the first term loss function representing the discriminant model is a two-class cross entropy loss function: i represents each layer of the discriminant model, for a total of m layers; j represents j points in the grid obtained from the ith layer, and n is the total of each layeriCounting; y isi，jRepresenting the true probability, p, of the presence of the ith point of the ith layeri，jRepresenting the prediction probability of the j point of the ith layer;
Figure BDA0003273663140000036
the second term loss function representing the discriminant model is a multi-class cross entropy loss function: n refers to n points in the completed point cloud, k is the total number of categories, yi,cMeans the true probability, p, that the ith point belongs to class ci,cPredicting probability that the predicted ith point belongs to the class c;
Figure BDA0003273663140000037
Figure BDA0003273663140000038
representing the loss function of the generative model, omega being the entire three-dimensional space, omega0For the surface of an object, the function phi is a sign of the generative model representationA distance function, x being a three-dimensional coordinate point,
Figure BDA0003273663140000039
n (x) is the normal vector to coordinate point x, ψ (φ (x)) -exp (- α | φ (x) |), α>>1。
The invention also provides a super-resolution device of the laser radar, which comprises:
the shape embedding determining module is used for inputting the original point cloud into the discriminant model to obtain the shape embedding of the point cloud block corresponding to the original point cloud; the original point cloud is obtained by constraining point cloud data of the laser radar obtained by scanning;
the system comprises a symbol distance function value determining module, a shape embedding module and a generating model, wherein the symbol distance function value determining module is used for splicing the three-dimensional coordinates of target grid points of target resolution and corresponding shape embedding values and inputting the three-dimensional coordinates into the generating model to obtain the symbol distance function value of the target grid points; the shape embedding value corresponding to the target grid point is obtained through linear interpolation based on the shape embedding of the point cloud block; the symbol distance function value is used for forming a point cloud result after super-resolution reconstruction;
the discriminant model and the generative model are obtained by performing collaborative training on the point cloud sample data and the corresponding reconstructed point cloud sample data.
The super-resolution apparatus for a lidar according to the present invention further includes:
a grid point screening module, configured to select a grid point for optimization completion based on a comparison result between an absolute value of a symbol distance function value of the target grid point and a preset threshold;
and the super-resolution reconstruction module is used for carrying out super-resolution reconstruction on the target point cloud based on the grid points for optimizing and completing to obtain the optimized target point cloud.
The super-resolution device of the laser radar of the present invention further comprises a classification module, wherein the classification module is configured to:
inputting the original point cloud into the discriminant model to obtain a point cloud which is completed under the same corresponding resolution and has a category probability value;
and the discriminant model is obtained by training point cloud sample data and the corresponding reconstructed point cloud class sample.
The present invention also provides an electronic device, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the super-resolution method of lidar as described in any of the above when executing the program.
The invention also provides a non-transitory computer-readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of the super-resolution method of lidar as described in any of the above.
The invention combines the generative model and the discriminant model, and realizes the rapid super-resolution reconstruction process of new laser radar data by learning the existing high-resolution data. Because the symbolic distance function of the scene object is modeled, the symbolic distance function for reconstructing a result to any resolution can be predicted, and the continuous representation of the object surface is realized, so that the super-resolution reconstruction of the laser radar point cloud data can be realized under any resolution; by learning information in a large amount of scene data, the completion process further utilizes big data prior knowledge on the basis of utilizing a geometric principle, so that a superior super-resolution reconstruction result compared with the prior art is obtained; target point cloud data obtained by scanning of the laser radar can be rapidly supplemented, and the resolution ratio can be improved, so that the effect of feeding back and optimizing a finished point cloud result in real time is realized; the invention can realize super-resolution reconstruction by using the simple electronic equipment and the storage medium and executing the program to process point cloud data without the support of complex hardware equipment, so the invention can be directly applied to the existing system using laser radar equipment and has strong transportability.
Drawings
In order to more clearly illustrate the technical solutions of the present invention or the prior art, the drawings needed for the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and those skilled in the art can also obtain other drawings according to the drawings without creative efforts.
FIG. 1 is a schematic flow chart of a super-resolution method for a lidar according to the present invention;
FIG. 2 is a schematic structural diagram of a super-resolution device of a laser radar according to the present invention;
fig. 3 is a schematic structural diagram of an electronic device provided in the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is obvious that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The super-resolution method of the laser radar of the present invention is described below with reference to fig. 1, and the method includes:
s1, inputting the original point cloud into a discriminant model to obtain the shape embedding of a point cloud block corresponding to the original point cloud; the original point cloud is obtained by constraining point cloud data of the laser radar obtained by scanning;
the original point cloud is used for generating shape embedding of point cloud blocks corresponding to the original point cloud. And (3) constraining the scanned laser radar point cloud data to obtain an original point cloud containing a plurality of point cloud blocks. The constraints may specifically be: and constraining point cloud data obtained by scanning the laser radar to a specified cuboid range, wherein the range is a part which needs to be concerned in an actual application scene.
In the discriminant model: the first half part uses a U-Net structure and is connected with a plurality of convolutional neural network layers, the sparse point cloud is completed and each point is classified, and a classification result is output; the second half part is connected with a plurality of convolutional neural network layers, and finally the shape of the output point cloud block is embedded. In this case, the discriminant model has the following input and output forms: inputting 'laser radar point cloud after constraint'; outputting 'the supplemented point cloud with the category probability value under the same corresponding resolution' and 'the embedding of the shape of the point cloud block'.
S2, splicing the three-dimensional coordinates of the target grid points of the target resolution and the corresponding shape embedded values thereof together and inputting the spliced three-dimensional coordinates into a generative model to obtain a symbol distance function value of the target grid points; the shape embedding value corresponding to the target grid point is obtained through linear interpolation based on the shape embedding of the point cloud block; the generative model is a neural network formed by a plurality of fully connected layers; the symbol distance function value is used for forming a point cloud result after super-resolution reconstruction;
when inputting these points into the generative model, instead of inputting the three-dimensional coordinates (x, y, z values) of the points, the coordinates are pieced together in the form of "predicting the shape of the point cloud block from the discriminant model and embedding the shape of each target grid point by linear interpolation", that is, the following forms are input: (S)1,S2,...,SnX, y, z) in which (S)1,S2,...,Sn) Vectors are embedded for the shapes corresponding to the (x, y, z) points, for a total of n dimensions.
The final output of the training process is: and inputting the symbol distance function value corresponding to each point in the generative model.
The discriminant model and the generative model are obtained by performing collaborative training on the point cloud sample data and the corresponding reconstructed point cloud sample data. In the training process, a method combining self-supervision training and supervision training is used, and a discriminant model for embedding the prediction shape and a generative model for fitting a symbolic distance function are optimized in a gradient descent and back propagation mode.
Specifically, in the training phase: firstly, inputting the constrained original point cloud into a discriminant model, and mapping the original sparse point cloud scanned by each frame of laser radar to corresponding shapes of point cloud data with uniform distribution and low sparsity degree by the network for embedding. And then embedding and inputting the points in the optimized reconstruction point cloud corresponding to the frame, the points randomly obtained in the constrained space and the shapes corresponding to each point into a generative model, and performing fitting training on the generative model. In the generative model, multi-frame point cloud data are trained simultaneously, and all point clouds share the same generative model in the training process. In order to distinguish the symbol distance functions of different point clouds, the three-dimensional coordinates of the point clouds are spliced with shape embedded values representing the scene and the position of the point in the input process. Because each frame of point cloud has many similar scenes, each scene is uniformly divided into a plurality of point cloud blocks when a discriminant model for predicting shape embedding is trained, and each block is respectively endowed with a shape embedding value. And (3) obtaining a shape embedded value corresponding to each point in the optimized and reconstructed point cloud provided by the training data in a linear interpolation mode, and then inputting the shape embedded value into the generative model for training. Since the entire computational process is conducted, training can be performed in the end-to-end manner described above.
The three-dimensional coordinates input into the generative model can be refined by means of encoding. I.e. mapping the three-dimensional coordinates to the m-dimension. Thus, the input of the generative model becomes (m + n) -dimensional data. Wherein m-dimensional coded coordinates and n-dimensional shapes are embedded. I.e. x, y, z are coded as:
(sin(πx),cos(πx),sin(2πx),cos(2πx),……)
(sin(πy),cos(πy),sin(2πy),cos(2πy),……)
(sin(πz),cos(πz),sin(2πz),cos(2πz),……)
each in m/3 dimensions.
The method combines a generating model and a discriminant model, realizes the rapid super-resolution reconstruction process of new laser radar data by learning the existing high-resolution data, and can realize the super-resolution reconstruction of the laser radar point cloud data on any resolution ratio as the symbolic distance function of a scene object is modeled and the continuous characterization of the object surface is realized; the characteristic enables a user to autonomously select the reconstructed resolution according to an application scene without complex algorithm modification and hardware equipment replacement. Meanwhile, the symbolic distance function of the object is modeled, so that the object rendering is possible, and more application requirements are met.
According to the invention, by learning the information in the large batch of scene data, the prior knowledge of the big data is further utilized on the basis of utilizing the geometric principle in the completion process, so that a better super-resolution reconstruction result than that in the prior art is obtained; target point cloud data obtained by scanning of the laser radar can be supplemented quickly, and the resolution ratio is improved, so that the effect of feeding back and optimizing the point cloud result in real time is realized; the invention can realize super-resolution reconstruction by using the simple electronic equipment and the storage medium and executing the program to process point cloud data without the support of complex hardware equipment, so the invention can be directly applied to the existing system using laser radar equipment and has strong portability.
The super-resolution method for the laser radar according to the present invention includes, after the three-dimensional coordinates of the target grid points and the corresponding shape embedding values are spliced together and input into the generative model, obtaining a symbol distance function value of the target grid points:
and selecting the grid points for optimizing completion based on the comparison result of the absolute value of the symbol distance function value of the target grid point and a preset threshold. All the obtained grid points for optimizing completion can form a point cloud result after super-resolution reconstruction.
That is, after the value of the sign distance function is obtained, a small preset threshold (the preset threshold may be set to a number close to zero, for example, 0.001, 0.01, etc.) may be defined, and a point at which the value of the sign distance function is smaller in absolute value than the threshold may be regarded as a point at which the value of the sign distance function is zero, that is, a point on the surface of the object, and other points may be regarded as points not on the surface. Therefore, the results obtained after the optimization improvement and super-resolution reconstruction of the input point cloud are predicted.
The super-resolution method for the laser radar according to the present invention is a method for selecting a grid point for optimization completion based on a comparison result between an absolute value of a function value of a symbol distance of the target grid point and a preset threshold, including:
comparing the absolute value of the symbol distance function value of the target grid point with a preset threshold value to obtain a comparison result;
and if the comparison result is that the absolute value of the symbol distance function value of the target grid point is smaller than a preset threshold value, taking the target grid point as a grid point for optimizing completion.
If the comparison result is that the absolute value of the symbol distance function value of the target grid point is not smaller than the preset threshold, the target grid point is regarded as a grid point which is not on the surface of the object, and the grid point which is not on the surface of the object can be discarded without consideration.
The super-resolution method of the laser radar according to the present invention, wherein the method further comprises:
inputting the original point cloud into the discriminant model to obtain a point cloud which is completed under the same corresponding resolution and has a category probability value;
and the discriminant model is obtained by training point cloud sample data and the corresponding reconstructed point cloud class sample.
During super-resolution reconstruction, sparse point clouds are input into a super-resolution reconstruction network, and a reconstructed point cloud A under any predicted resolution and a reconstructed point cloud B with category information under the resolution corresponding to the input point cloud are obtained. Next, for each point a in a, a point B closest to a is found in B by nearest neighbor search, and the point a is given the category information of the point B. Thus, each point in A has category information. And finally obtaining a super-resolution point cloud reconstruction result with category information under any resolution.
Since the output of the discriminant model is connected to the generative model, the whole calculation process is conductive, so that the method can be realized byAnd the optimization process of the loss function realizes the simultaneous training of the discriminant model and the generative model. The super-resolution method of the laser radar is characterized in that the discriminant model and the generative model are cooperatively trained loss functions
Figure BDA0003273663140000101
Comprises the following steps:
Figure BDA0003273663140000102
Figure BDA0003273663140000103
Figure BDA0003273663140000104
wherein,
Figure BDA0003273663140000105
the first term loss function representing the discriminant model is a two-class cross entropy loss function: (corresponding training process is the training process of embedding the shape of the output point cloud block), i represents each layer of the discriminant model, and the total m layers; j represents j points in the grid obtained from the ith layer, and n is the total of each layeriPoint; y isi，jRepresenting the true probability, p, of the presence of the ith point of the ith layeri，jRepresenting the prediction probability of the j point of the ith layer;
Figure 1
the second loss function representing the discriminant model is a multi-classification cross entropy loss function (the corresponding training process is that of the complemented point cloud with class probability value under the same resolution corresponding to the output point cloud block), n refers to n points in the complemented point cloud, k is the total number of classes, y is the total number of classesi，cRefers to the true probability that the ith point belongs to class c (for each point i, there is and only isThe true probability of one class is 1, the true probability of the other class is 0, that is, the true class of the point can only be a certain class), pi，cPredicting probability that the predicted ith point belongs to the class c;
in the discriminant model: the first half part uses a U-Net structure and is connected with a plurality of convolutional neural network layers, the sparse point cloud is completed and each point is classified, and a classification result is output; the second half part is connected with a plurality of convolutional neural network layers, and finally the shape of the output point cloud block is embedded. At this time, our discriminant model has the following input and output forms: inputting 'laser radar point cloud after constraint'; outputting 'point clouds with category probability values after completion under the same corresponding resolution' and 'embedding the shapes of point cloud blocks'. The processes of classifying and generating the shape embedding of the point cloud blocks are carried out synchronously and are also trained together in a training phase.
Figure BDA0003273663140000111
Figure BDA0003273663140000112
Representing the loss function of the generative model, omega being the entire three-dimensional space, omega0Is the surface of an object, the function phi is a symbol distance function represented by a generative model, x is a three-dimensional coordinate point,
Figure BDA0003273663140000113
n (x) is a normal vector of the coordinate point x, ψ (φ (x)) -exp (- α | φ (x) |), α>>1. While minimizing the loss: the first integral term in the formula makes the mode length of the gradient of the symbolic distance function in the whole space tend to 1 (the characteristic of the symbolic distance function in the space field is reflected physically); the first addition term in the second integral term makes the value of the sign distance function of a point on the surface tend to 0 (physically reflecting the distance of a point on the surface to the surface as 0), and the second addition term makes the gradient of a point on the surface and the normal vector of the point tend to be in the same direction (physically, the gradient is in the same direction)Representing the direction of the fastest function value change, and for a symbol distance function, the direction of a surface normal vector is just the direction of the fastest function value change); the third integral term makes use of the exp exponential function so that the value of the sign distance function for points not on the surface tends to infinity.
In the generative model, when training with the loss function: for each frame of corresponding points (on the surface of the object) in the optimized reconstruction point cloud, optimizing the predicted sign distance function values of the points to zero; for points randomly taken within the constrained space (not on the object surface), the predicted symbol distance function values for them can be optimized to their true symbol distance function values, but since it is the object surface that is of interest in the super-resolution reconstruction process, i.e., the point where the symbol distance function value is zero, the predicted symbol distance function values for these non-surface points are chosen to be optimized to infinity, thereby enhancing the degree of discrimination for the surface.
For laser radar data, point cloud has larger sparsity. Although the super-resolution reconstruction can be directly performed on the original point cloud data by using only a single degenerate generative model, the obtained result has the problems of uneven distribution, unsatisfactory reconstruction result details, high data noise and the like. Therefore, in order to achieve better super-resolution reconstruction, it is necessary to compensate for the sparsity, i.e. to complete the sparse region. The mode of combining the generative model and the discriminant model in the invention enables the completion process to be carried out along with the super-resolution reconstruction process.
Referring to fig. 2, the super-resolution apparatus of a laser radar according to the present invention will be described below, and the super-resolution apparatus of a laser radar described below and the super-resolution method of a laser radar described above may be referred to in correspondence with each other, and the super-resolution apparatus of a laser radar includes:
a shape embedding determining module 10, configured to input the original point cloud into a discriminant model, and obtain a shape embedding of a point cloud block corresponding to the original point cloud; the original point cloud is obtained by constraining point cloud data of the laser radar obtained by scanning;
the original point cloud is used for generating shape embedding of a point cloud block corresponding to the original point cloud, and the original point cloud containing a plurality of point cloud blocks can be obtained by constraining the scanned laser radar point cloud data. The constraints may specifically be: firstly, point cloud data obtained by scanning of a laser radar is constrained to a specified cuboid range, and the range is a part needing attention in an actual application scene.
A sign distance function value determining module 20, configured to splice together the three-dimensional coordinates of the target grid points of the target resolution and the corresponding shape embedded values thereof, and input the resulting three-dimensional coordinates and the corresponding shape embedded values into a generative model to obtain a sign distance function value of the target grid points; the shape embedding value corresponding to the target grid point is obtained through linear interpolation based on the shape embedding of the point cloud block; the symbol distance function value is used for forming a point cloud result after super-resolution reconstruction;
and the discriminant model and the generative model are obtained by performing collaborative training on the point cloud sample data and the corresponding reconstructed point cloud sample data.
When inputting these points into the generative model, not only the three-dimensional coordinates (x, y, z values) of the points, but also the shape embedding value (i.e., shape embedding vector) of each target grid point obtained by "predicting the shape of the point cloud block from the discriminant model and then performing linear interpolation" in coordinate splicing is input in the form of: (S)1，S2，...，SnX, y, z) in which (S)1,S2,...，Sn) Vectors are embedded for the shapes corresponding to the (x, y, z) points, for a total of n dimensions.
The final output of the training process is: and inputting the sign distance function value corresponding to each point in the generative model.
The discriminant model and the generative model are obtained by performing collaborative training on the point cloud sample data and the corresponding reconstructed point cloud sample data. In the training process, a method combining self-supervision training and supervision training is used, and a discriminant model embedded with the predicted shape and a generative model fitting a symbolic distance function are optimized in a gradient descent and back propagation mode.
Specifically, in the training phase: firstly, inputting the constrained original point cloud into a discriminant model, and mapping the original sparse point cloud scanned by each frame of laser radar to corresponding shapes of point cloud data with uniform distribution and low sparsity degree by the network for embedding. And then embedding and inputting the points in the optimized reconstruction point cloud corresponding to the frame, the points randomly obtained in the constrained space and the shapes corresponding to each point into a generative model, and performing fitting training on the generative model. In the generative model, multi-frame point cloud data are trained simultaneously, and all point clouds share the same generative model in the training process. In order to distinguish the symbol distance functions of different point clouds, the three-dimensional coordinates of the point clouds are spliced with shape embedded values representing the scene and the position of the point in the input process. Because each frame of point cloud has many similar scenes, each scene is uniformly divided into a plurality of point cloud blocks when a discriminant model for predicting shape embedding is trained, and each block is respectively endowed with a shape embedding value. And (3) obtaining a shape embedded value corresponding to each point in the optimized and reconstructed point cloud provided by the training data in a linear interpolation mode, and then inputting the shape embedded value into the generative model for training. Since the entire computational process is conducted, training can be performed in the end-to-end manner described above.
The super-resolution device of a laser radar according to the present invention further includes:
and the super-resolution reconstruction module is used for selecting the grid points for optimizing completion based on the comparison result of the absolute value of the symbol distance function value of the target grid point and a preset threshold value so as to form a point cloud result after super-resolution reconstruction.
That is, after the value of the sign distance function is obtained, a smaller preset threshold (the preset threshold may be set to a number close to zero, such as 0.001, 0.01, etc.) may be defined, and a point where the absolute value of the sign distance function is smaller than the threshold may be regarded as a point where the value of the sign distance function is zero, that is, a point on the surface of the object, and other points may be regarded as points not on the surface. Therefore, the results obtained after the optimization improvement and super-resolution reconstruction of the input point cloud are predicted.
The super-resolution device of the laser radar of the present invention, wherein the super-resolution reconstruction module is specifically configured to:
comparing the absolute value of the symbol distance function value of the target grid point with a preset threshold value to obtain a comparison result;
and if the comparison result is that the absolute value of the symbol distance function value of the target grid point is smaller than a preset threshold value, taking the target grid point as a grid point for optimizing completion.
If the comparison result is that the absolute value of the symbol distance function value of the target grid point is not smaller than the preset threshold, the target grid point is regarded as a grid point which is not on the surface of the object, and the grid point which is not on the surface of the object can be discarded without consideration.
Because the output of the discriminant model is connected to the generative model, the whole calculation process is conductive, so that the discriminant model and the generative model can be simultaneously trained through the optimization process of the following loss functions. The super-resolution method of the laser radar according to the present invention, wherein,
loss function of collaborative training of discriminant model and generative model
Figure BDA0003273663140000141
Comprises the following steps:
Figure BDA0003273663140000142
Figure BDA0003273663140000143
Figure BDA0003273663140000144
wherein,
Figure BDA0003273663140000151
a first loss function of the discriminant model is expressed (the corresponding training process is the training process of embedding the shape of the output point cloud block), i expresses each layer of the discriminant model, and the total number is m; j represents j points in the grid obtained from the ith layer, and n is the total of each layeriCounting; y isi，jRepresenting the true probability, p, of the existence of the jth point of the ith layeri，jRepresenting the prediction probability of the j point of the ith layer;
Figure BDA0003273663140000152
a second loss function representing the discriminant model (the corresponding training process is that of the complemented point cloud with class probability value under the same resolution corresponding to the output point cloud block), n refers to n points in the complemented point cloud, k is the total number of classes, y is the total number of classesi,cThe true probability that the ith point belongs to the class c (for each point i, the true probability of one class is 1, the true probability of other classes is 0, namely the true class of the point can only be a certain class), pi,cThe predicted probability that the ith point belongs to the class c is set;
in the discriminant model: the first half part uses a U-Net structure and is connected with a plurality of convolutional neural network layers, the sparse point cloud is completed and each point is classified, and a classification result is output; the latter half is connected with a plurality of convolution neural network layers, and finally the shape of the output point cloud is embedded. At this time, our discriminant model has the following input and output forms: inputting 'laser radar point cloud after constraint'; outputting 'the supplemented point cloud with the category probability value under the same corresponding resolution' and 'the embedding of the shape of the point cloud block'. The processes of classifying and generating the shape embedding of the point cloud blocks are synchronously carried out and are also cooperatively trained together in the training phase.
Figure BDA0003273663140000153
Figure BDA0003273663140000154
Representing the loss function of the generative model, omega being the entire three-dimensional space, omega0Is the surface of an object, the function phi is a symbol distance function represented by a generative model, x is a three-dimensional coordinate point,
Figure BDA0003273663140000155
n (x) is the normal vector to coordinate point x, ψ (φ (x)) -exp (- α | φ (x) |), α>>1. While minimizing the loss: the first integral term in the formula enables the gradient mode length of the symbolic distance function in the whole space to be close to 1 (the characteristic of the symbolic distance function in the space field is reflected physically); the first addition term in the second integral term enables the sign distance function value of a point on the surface to tend to be 0 (physically reflecting that the distance from the point on the surface to the surface is 0), and the second addition term enables the gradient of the point on the surface and the normal vector of the point to tend to be in the same direction (physically, the gradient represents the direction in which the function value changes most rapidly, and for the sign distance function, the normal vector direction of the surface is just the direction in which the function value changes most rapidly); the third integral term makes use of the exp exponential function so that the value of the sign distance function for points not on the surface tends to infinity.
In the generative model, when training with the loss function: for each frame of corresponding points (on the surface of the object) in the optimized reconstruction point cloud, optimizing the predicted sign distance function values of the points to zero; for points taken randomly within the constrained space (not on the object surface), the predicted sign distance function values for them can be optimized to their true sign distance function values, but since it is the object surface that is of interest in the super-resolution reconstruction process, i.e. the point where the sign distance function value is zero, the predicted sign distance function values for these non-surface points are chosen to be optimized to infinity, thereby enhancing the degree of discrimination for the surface.
For laser radar data, point cloud has larger sparsity. Although the super-resolution reconstruction can be directly performed on the original point cloud data by using only a single degenerate generative model, the obtained result has the problems of uneven distribution, unsatisfactory reconstruction result details, high data noise and the like. Therefore, in order to achieve better super-resolution reconstruction, it is necessary to compensate for the sparsity, i.e. to complete the sparse region. The mode of combining the generative model and the discriminant model in the invention enables the completion process to be carried out along with the super-resolution reconstruction process.
The super-resolution device for the laser radar of the invention further comprises a classification module, wherein the classification module is used for:
inputting the original point cloud into the discriminant model to obtain a point cloud which is completed under the same corresponding resolution and has a category probability value;
and the discriminant model is obtained by training point cloud sample data and the corresponding reconstructed point cloud class sample.
Fig. 3 illustrates a physical structure diagram of an electronic device, which may include: a processor (processor)310, a communication Interface (communication Interface)320, a memory (memory)330 and a communication bus 340, wherein the processor 310, the communication Interface 320 and the memory 330 communicate with each other via the communication bus 340. Processor 310 may invoke logic instructions in memory 330 to perform a super resolution method for a lidar, the method comprising:
s1, inputting the original point cloud into a discriminant model to obtain the shape embedding of a point cloud block corresponding to the original point cloud; the original point cloud is obtained by constraining point cloud data of the laser radar obtained by scanning;
s2, splicing the three-dimensional coordinates of the target grid points of the target resolution and the corresponding shape embedded values thereof together and inputting the spliced three-dimensional coordinates into a generative model to obtain a symbol distance function value of the target grid points; the shape embedding value corresponding to the target grid point is obtained through linear interpolation based on the shape embedding of the point cloud block; the symbol distance function value is used for forming a point cloud result after super-resolution reconstruction;
the discriminant model and the generative model are obtained by performing collaborative training on the point cloud sample data and the corresponding reconstructed point cloud sample data.
In addition, the logic instructions in the memory 330 may be implemented in the form of software functional units and stored in a computer readable storage medium when the software functional units are sold or used as independent products. Based on such understanding, the technical solution of the present invention or a part thereof which substantially contributes to the prior art may be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
In another aspect, the present invention also provides a computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions which, when executed by a computer, enable the computer to perform the super-resolution method of lidar provided by the above methods, the method comprising:
s1, inputting the original point cloud into a discriminant model to obtain the shape embedding of a point cloud block corresponding to the original point cloud; the original point cloud is obtained by constraining point cloud data of the laser radar obtained by scanning;
s2, splicing the three-dimensional coordinates of the target grid points of the target resolution and the corresponding shape embedded values thereof together and inputting the spliced three-dimensional coordinates into a generative model to obtain a symbol distance function value of the target grid points; the shape embedding value corresponding to the target grid point is obtained through linear interpolation based on the shape embedding of the point cloud block; the symbol distance function value is used for forming a point cloud result after super-resolution reconstruction;
and the discriminant model and the generative model are obtained by performing collaborative training on the point cloud sample data and the corresponding reconstructed point cloud sample data.
In still another aspect, the present invention also provides a non-transitory computer-readable storage medium having stored thereon a computer program, which when executed by a processor, implements a super-resolution method for a lidar to perform the above-mentioned methods, the method comprising:
s1, inputting the original point cloud into a discriminant model to obtain the shape embedding of a point cloud block corresponding to the original point cloud; the original point cloud is obtained by constraining point cloud data of the laser radar obtained by scanning;
s2, splicing the three-dimensional coordinates of the target grid points of the target resolution and the corresponding shape embedded values thereof together and inputting the spliced three-dimensional coordinates into a generative model to obtain a symbol distance function value of the target grid points; the shape embedding value corresponding to the target grid point is obtained through linear interpolation based on the shape embedding of the point cloud block; the symbol distance function value is used for forming a point cloud result after super-resolution reconstruction;
the discriminant model and the generative model are obtained by performing collaborative training on the point cloud sample data and the corresponding reconstructed point cloud sample data.
The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware. With this understanding in mind, the above-described technical solutions may be embodied in the form of a software product, which can be stored in a computer-readable storage medium such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in the embodiments or some parts of the embodiments.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.

Claims (10)

1. A super-resolution method of a laser radar is characterized by comprising the following steps:
inputting the original point cloud into a discriminant model to obtain the shape embedding of a point cloud block corresponding to the original point cloud; the original point cloud is obtained by constraining point cloud data of the laser radar obtained by scanning;
splicing the three-dimensional coordinates of target grid points of a target resolution and corresponding shape embedding values thereof together and inputting the spliced three-dimensional coordinates into a generative model to obtain a symbol distance function value of the target grid points; the shape embedding value corresponding to the target grid point is obtained through linear interpolation based on the shape embedding of the point cloud block; the symbol distance function value is used for forming a point cloud result after super-resolution reconstruction;
and the discriminant model and the generative model are obtained by performing collaborative training on the point cloud sample data and the corresponding reconstructed point cloud sample data.
2. The super-resolution method for lidar according to claim 1, wherein the stitching together the three-dimensional coordinates of the target grid points and the corresponding shape embedding values and inputting the stitched coordinates into the generative model to obtain the symbol distance function values of the target grid points comprises:
and selecting the grid points for optimizing completion based on the comparison result of the absolute value of the symbol distance function value of the target grid point and a preset threshold value so as to form a point cloud result after super-resolution reconstruction.
3. The super-resolution method for lidar according to claim 2, wherein the selecting a grid point for optimal completion based on a comparison result between an absolute value of the symbol distance function of the target grid point and a preset threshold comprises:
comparing the absolute value of the symbol distance function value of the target grid point with a preset threshold value to obtain a comparison result;
and if the comparison result is that the absolute value of the symbol distance function value of the target grid point is smaller than a preset threshold value, taking the target grid point as a grid point for optimizing completion.
4. The super resolution method for lidar according to claim 1, wherein the method further comprises:
inputting the original point cloud into the discriminant model to obtain a point cloud which is completed under the same corresponding resolution and has a category probability value;
and the discriminant model is obtained by training point cloud sample data and the corresponding reconstructed point cloud class sample.
5. The super-resolution method for lidar according to claim 4, wherein the discriminant model and the generative model are loss functions trained cooperatively
Figure FDA0003273663130000021
Comprises the following steps:
Figure FDA0003273663130000022
Figure FDA0003273663130000023
Figure FDA0003273663130000024
wherein,
Figure FDA0003273663130000025
the first term loss function representing the discriminant model is a two-class cross entropy loss function: i represents each layer of the discriminant model, for a total of m layers; j represents j points in the grid obtained from the ith layer, and n is the total of each layeriPoint; y isi，jRepresenting the true probability, p, of the existence of the jth point of the ith layeri，jRepresenting the prediction probability of the j point of the ith layer;
Figure FDA0003273663130000026
the second term loss function representing the discriminant model is a multi-class cross entropy loss function: n refers to n points in the completed point cloud, k is the total number of categories, yi,cMeans the true probability, p, that the ith point belongs to class ci,cPredicting probability that the predicted ith point belongs to the class c;
Figure FDA0003273663130000027
Figure FDA0003273663130000028
representing the loss function of the generative model, omega being the entire three-dimensional space, omega0Is the surface of an object, boxPhi is a symbol distance function represented by the generative model, x is a three-dimensional coordinate point,
Figure FDA0003273663130000029
n (x) is the normal vector to coordinate point x, ψ (φ (x)) -exp (- α | φ (x) |), α>>1。
6. A super-resolution device for a laser radar, comprising:
the shape embedding determining module is used for inputting the original point cloud into the discriminant model to obtain the shape embedding of the point cloud block corresponding to the original point cloud; the original point cloud is obtained by constraining point cloud data of the laser radar obtained by scanning;
the system comprises a symbol distance function value determining module, a shape embedding module and a generating model, wherein the symbol distance function value determining module is used for splicing the three-dimensional coordinates of target grid points of target resolution and corresponding shape embedding values and inputting the three-dimensional coordinates into the generating model to obtain the symbol distance function value of the target grid points; the shape embedding value corresponding to the target grid point is obtained through linear interpolation based on the shape embedding of the point cloud block; the symbol distance function value is used for forming a point cloud result after super-resolution reconstruction;
and the discriminant model and the generative model are obtained by performing collaborative training on the point cloud sample data and the corresponding reconstructed point cloud sample data.
7. The super-resolution device for lidar according to claim 6, wherein the super-resolution device for lidar further comprises:
and the super-resolution reconstruction module is used for selecting the grid points for optimizing completion based on the comparison result of the absolute value of the symbol distance function value of the target grid point and a preset threshold value so as to form a point cloud result after super-resolution reconstruction.
8. The lidar super-resolution device according to claim 6, further comprising a classification module configured to:
inputting the original point cloud into the discriminant model to obtain a point cloud which is completed under the same corresponding resolution and has a category probability value;
and the discriminant model is obtained by training based on the point cloud sample data and the corresponding reconstructed point cloud class sample.
9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor, when executing the program, carries out the steps of the super resolution method of lidar according to any of claims 1 to 5.
10. A non-transitory computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the super-resolution method for lidar according to any one of claims 1 to 5.
CN202111109236.5A 2021-09-22 2021-09-22 Super-resolution method and device of laser radar, electronic equipment and storage medium Active CN114764746B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111109236.5A CN114764746B (en) 2021-09-22 2021-09-22 Super-resolution method and device of laser radar, electronic equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111109236.5A CN114764746B (en) 2021-09-22 2021-09-22 Super-resolution method and device of laser radar, electronic equipment and storage medium

Publications (2)

Publication Number Publication Date
CN114764746A true CN114764746A (en) 2022-07-19
CN114764746B CN114764746B (en) 2024-09-06

Family

ID=82365187

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111109236.5A Active CN114764746B (en) 2021-09-22 2021-09-22 Super-resolution method and device of laser radar, electronic equipment and storage medium

Country Status (1)

Country Link
CN (1) CN114764746B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116128727A (en) * 2023-02-02 2023-05-16 中国人民解放军国防科技大学 Super-resolution method, system, equipment and medium for polarized radar image

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109493407A (en) * 2018-11-19 2019-03-19 腾讯科技（深圳）有限公司 Realize the method, apparatus and computer equipment of laser point cloud denseization
US20200158869A1 (en) * 2018-11-19 2020-05-21 Elmira Amirloo Abolfathi System, device and method of generating a high resolution and high accuracy point cloud
CN112561796A (en) * 2020-12-02 2021-03-26 西安电子科技大学 Laser point cloud super-resolution reconstruction method based on self-attention generation countermeasure network

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109493407A (en) * 2018-11-19 2019-03-19 腾讯科技（深圳）有限公司 Realize the method, apparatus and computer equipment of laser point cloud denseization
US20200158869A1 (en) * 2018-11-19 2020-05-21 Elmira Amirloo Abolfathi System, device and method of generating a high resolution and high accuracy point cloud
CN112561796A (en) * 2020-12-02 2021-03-26 西安电子科技大学 Laser point cloud super-resolution reconstruction method based on self-attention generation countermeasure network

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
TAKAYUKI SHINOHARA等: "Point2Wave: 3-D Point Cloud to Waveform Translation Using a Conditional Generative Adversarial Network With Dual Discriminators", IEEE, 2 September 2021 (2021-09-02) *
龚道然 等: "激光雷达三维距离像超分辨重构方法研究", 《红外与激光工程》, vol. 49, no. 8, 31 August 2020 (2020-08-31) *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116128727A (en) * 2023-02-02 2023-05-16 中国人民解放军国防科技大学 Super-resolution method, system, equipment and medium for polarized radar image

Also Published As

Publication number Publication date
CN114764746B (en) 2024-09-06

Similar Documents

Publication Publication Date Title
Wang et al. Hf-neus: Improved surface reconstruction using high-frequency details
Jam et al. A comprehensive review of past and present image inpainting methods
CN112529015B (en) Three-dimensional point cloud processing method, device and equipment based on geometric unwrapping
Wynn et al. Diffusionerf: Regularizing neural radiance fields with denoising diffusion models
CN111325851B (en) Image processing method and device, electronic equipment and computer readable storage medium
WO2021027759A1 (en) Facial image processing
CN111753698B (en) Multi-mode three-dimensional point cloud segmentation system and method
Ghorai et al. Multiple pyramids based image inpainting using local patch statistics and steering kernel feature
CN111028327A (en) Three-dimensional point cloud processing method, device and equipment
US12026892B2 (en) Figure-ground neural radiance fields for three-dimensional object category modelling
CN118202391A (en) Neural radiation field-generating modeling of object classes from a single two-dimensional view
US20230215085A1 (en) Object-centric neural decomposition for image re-rendering
US20240119697A1 (en) Neural Semantic Fields for Generalizable Semantic Segmentation of 3D Scenes
JP2023533907A (en) Image processing using self-attention-based neural networks
Ahmad et al. 3D capsule networks for object classification from 3D model data
CN112528811A (en) Behavior recognition method and device
CN113570573A (en) Pulmonary nodule false positive eliminating method, system and equipment based on mixed attention mechanism
EP4392935A1 (en) Robustifying nerf model novel view synthesis to sparse data
Wang et al. Improved surface reconstruction using high-frequency details
Jain et al. Adaptive & multi-resolution procedural infinite terrain generation with diffusion models and Perlin noise
CN114764746A (en) Super-resolution method and device for laser radar, electronic device and storage medium
JP2018124990A (en) Model generation apparatus, evaluation apparatus, model generation method, evaluation method, and program
JP7337303B2 (en) LEARNING DEVICE AND LEARNING METHOD
CN110490235B (en) Vehicle object viewpoint prediction and three-dimensional model recovery method and device facing 2D image
CN112184611A (en) Image generation model training method and device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant