CN115457206A - Three-dimensional model generation method, device, equipment and storage medium - Google Patents
Three-dimensional model generation method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN115457206A CN115457206A CN202211127231.XA CN202211127231A CN115457206A CN 115457206 A CN115457206 A CN 115457206A CN 202211127231 A CN202211127231 A CN 202211127231A CN 115457206 A CN115457206 A CN 115457206A
- Authority
- CN
- China
- Prior art keywords
- target
- voxel
- dimensional model
- original image
- pixels
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T15/00—3D [Three Dimensional] image rendering
- G06T15/005—General purpose rendering architectures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/90—Determination of colour characteristics
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computer Graphics (AREA)
- Artificial Intelligence (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Geometry (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Health & Medical Sciences (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Image Generation (AREA)
Abstract
The invention discloses a three-dimensional model generation method, a three-dimensional model generation device, three-dimensional model generation equipment and a storage medium, and belongs to the technical field of three-dimensional modeling. The method comprises the steps of obtaining an input original image set containing an object image to be reconstructed; performing image segmentation on the original image set to obtain target pixels and non-target pixels contained in each original image in the original image set; processing the target pixels and the non-target pixels, and obtaining a plurality of reference image sets according to the processed pixels; training each reference image set respectively to obtain a voxel color and a voxel density corresponding to each reference image set; generating a target three-dimensional model of the object to be reconstructed according to the voxel color, the voxel density and the density correction coefficient, effectively distinguishing target voxels from non-target voxels based on the color and the voxel density of the nerve radiation field obtained according to the reference model, and introducing the voxel density correction coefficient, thereby removing wrong suspended voxels and quickly optimizing the rendering effect.
Description
Technical Field
The invention relates to the technical field of three-dimensional modeling, in particular to a three-dimensional model generation method, a three-dimensional model generation device, three-dimensional model generation equipment and a storage medium.
Background
Nervus radiation field NeRF is one of the mainstream three-dimensional modeling techniques in the industry at present. The three-dimensional characteristics of the object are restored through information such as multi-angle pictures, and a virtual rendering image under a new view angle can be generated. The method has wide application in the field of AR/VR/analog simulation.
The traditional NeRF only uses image information, does not consider three-dimensional shape constraint, and is easy to form wrong color voxels suspended in space to influence modeling precision. The depth of field of an object is considered by the enhanced NeRF, but continuous dense images are needed to provide feature points to calculate three-dimensional coordinates, or equipment such as laser radars and the like are needed to provide point clouds with the three-dimensional coordinates, so that the use cost of the NeRF is increased. In addition, the traditional NeRF depends on the quality of an input image and a training model, and rendering results are difficult to adjust.
The above is only for the purpose of assisting understanding of the technical solution of the present invention, and does not represent an admission that the above is the prior art.
Disclosure of Invention
The invention mainly aims to provide a three-dimensional model generation method, a three-dimensional model generation device, three-dimensional model generation equipment and a storage medium, and aims to solve the technical problems that in the prior art, the construction cost of a three-dimensional model is high, and the rendering effect is difficult to adjust.
In order to achieve the above object, the present invention provides a three-dimensional model generation method, including the steps of:
acquiring an input original image set containing an object image to be reconstructed;
performing image segmentation on the original image set to obtain target pixels and non-target pixels contained in each original image in the original image set;
processing the target pixels and the non-target pixels, and obtaining a plurality of reference image sets according to the processed pixels;
training each reference image set respectively to obtain a voxel color and a voxel density corresponding to each reference image set;
and generating a target three-dimensional model of the object to be reconstructed according to the voxel colors and the voxel density and density correction coefficients.
Optionally, the processing the target pixels and the non-target pixels, and obtaining a plurality of reference image sets according to the processed pixels includes:
reserving target pixels contained in each original image;
filling non-target pixels contained in each original image according to a preset color;
and obtaining a plurality of reference images corresponding to each original image according to the target pixels and the filled non-target pixels.
Optionally, the preset colors include red, green and blue;
the filling of the non-target pixels included in each original image according to the preset color includes:
filling non-target pixels included in each original image with the red, the green, and the blue, respectively.
Optionally, the filled non-target pixels are a red pixel, a green pixel and a blue pixel respectively;
the obtaining a plurality of reference images corresponding to each original image according to the target pixels and the filled non-target pixels comprises:
obtaining red filling reference images corresponding to the original images according to the target pixels and the red pixels;
obtaining a green filling reference image corresponding to each original image according to the target pixel and the green pixel;
and obtaining a blue filling reference image corresponding to each original image according to the target pixel and the blue pixel.
Optionally, the generating a target three-dimensional model of the object to be reconstructed according to the voxel colors and the voxel density and density correction coefficients includes:
determining a target voxel color according to the voxel color;
calculating a target voxel density according to the voxel density and the density correction coefficient;
and generating a target three-dimensional model of the object to be reconstructed according to the target voxel color and the target voxel density.
Optionally, before determining the target voxel color according to the voxel color, the method further includes:
acquiring a target input vector, wherein the target input vector comprises a space point x position in a space where the object to be reconstructed is located and a view direction d of the original image or the reference image corresponding to the space point x;
and determining the voxel color and the voxel density corresponding to each reference image set through the target input vector.
Optionally, before calculating the target voxel density according to the voxel density and the density correction factor, the method further includes:
and determining a density correction coefficient according to the voxel color, the target voxel color and a preset constant.
In addition, to achieve the above object, the present invention also provides a three-dimensional model generation apparatus, including:
the acquisition module is used for acquiring an input original image set containing an object image to be reconstructed;
the identification module is used for carrying out image segmentation on the original image set to obtain target pixels and non-target pixels contained in each original image in the original image set;
the processing module is used for processing the target pixels and the non-target pixels and obtaining a plurality of reference image sets according to the processed pixels;
the training module is used for respectively training each reference image set to obtain the voxel color and the voxel density corresponding to each reference image set;
and the reconstruction module is used for generating a target three-dimensional model of the object to be reconstructed according to the voxel color and the voxel density and density correction coefficient.
Further, to achieve the above object, the present invention also proposes a three-dimensional model generation device including: a memory, a processor, and a three-dimensional model generation program stored on the memory and run on the processor, the three-dimensional model generation program configured to implement the three-dimensional model generation method as described above.
Furthermore, to achieve the above object, the present invention also proposes a storage medium having stored thereon a three-dimensional model generation program which, when executed by a processor, implements the three-dimensional model generation method as described above.
The method comprises the steps of obtaining an input original image set containing an object image to be reconstructed; performing image segmentation on the original image set to obtain target pixels and non-target pixels contained in each original image in the original image set; processing the target pixels and the non-target pixels, and obtaining a plurality of reference image sets according to the processed pixels; respectively training each reference image set to obtain a voxel color and a voxel density corresponding to each reference image set; and generating a target three-dimensional model of the object to be reconstructed according to the voxel color, the voxel density and the density correction coefficient, effectively distinguishing a target voxel from a non-target voxel on the basis of the color and the voxel density obtained by the nerve radiation field according to the reference model, and introducing the voxel density correction coefficient, so that the wrong suspended voxel is removed, and the rendering effect can be quickly optimized.
Drawings
FIG. 1 is a schematic structural diagram of a three-dimensional model generation device of a hardware operating environment according to an embodiment of the present invention;
FIG. 2 is a schematic flow chart of a three-dimensional model generation method according to a first embodiment of the present invention;
FIG. 3 is a schematic diagram of a camera shooting according to an embodiment of the three-dimensional model generation method of the present invention;
FIG. 4 is a schematic diagram of background interference during a camera shooting process according to an embodiment of the three-dimensional model generation method of the present invention;
FIG. 5 is a schematic diagram of a three-dimensional model construction process according to an embodiment of the three-dimensional model generation method of the present invention;
FIG. 6 is a schematic flow chart of a three-dimensional model generation method according to a second embodiment of the present invention;
FIG. 7 is a schematic flow chart of a three-dimensional model generation method according to a third embodiment of the present invention;
FIG. 8 is a schematic diagram of a target input vector in an embodiment of a three-dimensional model generation method according to the invention;
fig. 9 is a block diagram showing the configuration of the three-dimensional model generating apparatus according to the first embodiment of the present invention.
The implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.
Detailed Description
It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not limit the invention.
Referring to fig. 1, fig. 1 is a schematic structural diagram of a three-dimensional model generation device for a hardware operating environment according to an embodiment of the present invention.
As shown in fig. 1, the three-dimensional model generating apparatus may include: a processor 1001, such as a Central Processing Unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Wherein a communication bus 1002 is used to enable connective communication between these components. The user interface 1003 may include a Display (Display), an input unit such as a Keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface, a wireless interface. The network interface 1004 may optionally include a standard wired interface, a Wireless interface (e.g., a Wireless-Fidelity (Wi-Fi) interface). The Memory 1005 may be a Random Access Memory (RAM) Memory, or a Non-Volatile Memory (NVM), such as a disk Memory. The memory 1005 may alternatively be a storage device separate from the processor 1001.
Those skilled in the art will appreciate that the configuration shown in FIG. 1 does not constitute a limitation of a three-dimensional model generation apparatus and may include more or fewer components than those shown, or some components in combination, or a different arrangement of components.
As shown in fig. 1, a memory 1005, which is a storage medium, may include therein an operating system, a network communication module, a user interface module, and a three-dimensional model generation program.
In the three-dimensional model generation device shown in fig. 1, the network interface 1004 is mainly used for data communication with a network server; the user interface 1003 is mainly used for data interaction with a user; the processor 1001 and the memory 1005 in the three-dimensional model generation device of the present invention may be provided in a three-dimensional model generation device that calls the three-dimensional model generation program stored in the memory 1005 through the processor 1001 and executes the three-dimensional model generation method provided by the embodiment of the present invention.
An embodiment of the present invention provides a three-dimensional model generation method, and referring to fig. 2, fig. 2 is a schematic flow diagram of a first embodiment of a three-dimensional model generation method according to the present invention.
In this embodiment, the three-dimensional model generation method includes the following steps:
step S10: an input original image set containing an image of an object to be reconstructed is acquired.
In this embodiment, the execution subject of this embodiment may be the three-dimensional model generation device, which has functions of data processing, data communication, program execution, and the like, and the three-dimensional model generation device may be a terminal device such as a computer. Of course, other devices with similar functions may also be used, and the implementation condition is not limited thereto. For convenience of explanation, the present embodiment will be described by taking a three-dimensional model generating apparatus as an example.
It should be noted that nervus radiation field NeRF is one of the mainstream three-dimensional modeling techniques in the industry at present. The three-dimensional characteristics of the object are restored through information such as multi-angle pictures, and a virtual rendering image under a new view angle can be generated. The method has wide application in the field of AR/VR/analog simulation. Traditional NeRF uses only image information, does not consider three-dimensional shape constraints, and is easy to form wrong color voxels suspended in space, thereby affecting modeling accuracy. The depth of field of an object is considered by the enhanced NeRF, but continuous dense images are needed to provide feature points to calculate three-dimensional coordinates, or equipment such as laser radars and the like are needed to provide point clouds with the three-dimensional coordinates, so that the use cost of the NeRF is increased. In addition, the traditional NeRF depends on the quality of an input image and a training model, and rendering results are difficult to adjust. The input of the NeRF is object images observed from multiple visual angles, specifically, object pictures taken by one or more cameras from different angles, and a camera pose and camera internal parameters corresponding to each picture; the NeRF carries out position coding on the three-dimensional space, training is carried out through machine learning methods such as a multilayer perceptron, the imaging effect of the three-dimensional space at different visual angles is enabled to be as close as possible to the input image material, as shown in the right picture 3, cameras at two poses respectively shoot the same point S on an object, and then the NeRF can calculate the information such as the three-dimensional space position, the color, the transparency and the like of the point, so that the imaging effect in the two camera poses can be simultaneously met. However, it is inevitable that the camera-captured image contains, in addition to the object to be reconstructed, interference information such as background. When there is a scene as shown in fig. 4 on the right, just as the color at a in the background is close to the color at B on the object, neRF may consider that there are floating voxels at C that match this color, causing modeling errors. In order to solve the above technical problem, the rendering effect may be optimized quickly, and specifically, the rendering effect may be implemented as follows. In a specific implementation, in this embodiment, a three-dimensional model of an object is reconstructed, and first, an image set of the object to be reconstructed, that is, an input original image set needs to be obtained, where the original image set includes images of the object to be reconstructed observed from multiple viewing angles.
Further, in this embodiment, a three-dimensional model reconstruction flow in this embodiment is described by taking fig. 5 as an example. Firstly, in this embodiment, an original image set is obtained, then, the original image set is subjected to image segmentation to obtain a target image set including target pixels, then, non-target pixels in each original image are respectively filled according to red, green and blue, so that an image set filled with red, namely an R image set, an image set filled with green, namely a G image set, and an image set filled with blue, namely a B image set are obtained, then, each image set is trained through a multilayer perceptron to obtain an R model, a G model and a B model, and finally, a final target three-dimensional model is generated by combining a density coefficient and colors and densities corresponding to each model.
Step S20: and performing image segmentation on the original image set to obtain target pixels and non-target pixels contained in each original image in the original image set.
In a specific implementation, the original image set is formed by images from a plurality of viewing angles corresponding to a plurality of objects to be reconstructed, and in this embodiment, for each image, target pixels and non-target pixels included in the image are identified. In practical situations, the object to be reconstructed may be accompanied by a corresponding background, and in this embodiment, the object pixel and the non-object pixel included in each original image may be identified by segmenting the original image set, so as to distinguish the object to be reconstructed from the background. Specifically, in this embodiment, a pixel including the object to be reconstructed may be used as a target pixel, and a pixel not including the object to be reconstructed, that is, a background pixel, may be used as a non-target pixel.
Step S30: and processing the target pixels and the non-target pixels, and obtaining a plurality of reference image sets according to the processed pixels.
In a specific implementation, after the target pixels and the non-target pixels in each original image are identified, the identified target pixels and non-target pixels may be processed in this embodiment, and a plurality of reference image sets are obtained based on the processed pixels.
Specifically, the pixel processing method in this embodiment is to reserve the target pixel, fill the non-target pixels with other colors, that is, change the pixel values of the other non-target pixels, and then obtain a plurality of different reference images corresponding to one original image, thereby obtaining the most suitable reference imageThe final reference image set. For example, for the original image P, in the present embodiment, after the target pixel and the non-target pixel in the original image P are identified, the target pixel in the original image P is retained, and then the non-target pixels are respectively filled with red, green, and blue, so as to obtain three reference images P corresponding to the original image P 1 、P 2 And P 3 . Reference picture P 1 Including the target pixel and the filled red pixel, and the reference image P 2 Including the target pixel and the filled green pixel, the reference image P 3 The method comprises the steps of including target pixels and filled blue pixels, and adopting the mode for each original image, so that a red filled reference image, a green filled reference image and a blue filled reference image can be obtained finally.
Step S40: and respectively training each reference image set to obtain the voxel color and the voxel density corresponding to each reference image set.
In a specific implementation, after the reference image sets are obtained, in this embodiment, training may be performed on each reference image set, so as to obtain a voxel color and a voxel density corresponding to each reference image set. For example, assuming that the obtained reference image set is an R image set, a G image set and a B image set, training is performed on the R image set, so that the voxel color corresponding to the R image set is cr and the voxel density is σ R, training is performed on the G image set, so that the voxel color corresponding to the G image set is cg and the voxel density is σ G, and training is performed on the B image set, so that the voxel color corresponding to the B image set is cb and the voxel density is σ B. It should be noted that, in this embodiment, the reference image set may be trained through a NeRF neural radiation field model, where each of the R picture set, the G picture set, and the B picture set includes a target pixel and a filled non-target pixel, and cr, cg, and cb include RGB channel values corresponding to all voxels in the model.
Step S50: and generating a target three-dimensional model of the object to be reconstructed according to the voxel color and the voxel density and density correction coefficient.
It should be noted that, by introducing the density correction coefficient in this embodiment, a fast optimization rendering effect can be achieved.
In a specific implementation, after obtaining the voxel color and the voxel density corresponding to each reference image set, in this embodiment, a final target voxel color and a target voxel density are obtained based on the obtained voxel color and voxel density and by combining a density correction coefficient, and finally, a target three-dimensional model may be generated and obtained according to the target voxel color and the target voxel density.
The method comprises the steps of obtaining an input original image set containing an object image to be reconstructed; performing image segmentation on the original image set to obtain target pixels and non-target pixels contained in each original image in the original image set; processing the target pixels and the non-target pixels, and obtaining a plurality of reference image sets according to the processed pixels; training each reference image set respectively to obtain a voxel color and a voxel density corresponding to each reference image set; and generating a target three-dimensional model of the object to be reconstructed according to the voxel color, the voxel density and the density correction coefficient. The method comprises the steps of obtaining a color and a voxel density according to a reference model based on a nerve radiation field, effectively distinguishing a target voxel from a non-target voxel, introducing a voxel density correction coefficient, correcting the voxel density according to the variance of the voxel color, wherein the color variance of the target pixel is small, so that the voxel density corresponding to a target after multiplication with the voxel density coefficient is basically unchanged, but the color variance of the non-target pixel is large, and the non-target voxel density after multiplication with the voxel density coefficient is changed into a small value, so that an erroneous suspended voxel is removed, and the rendering effect can be quickly optimized.
Referring to fig. 6, fig. 6 is a flowchart illustrating a method for generating a three-dimensional model according to a second embodiment of the present invention.
Based on the first embodiment, in the three-dimensional model generating method according to this embodiment, the step S30 specifically includes:
step S301: the target pixel contained in each original image is retained.
It should be noted that, the image segmentation in this embodiment refers to distinguishing a target pixel from a non-target pixel, then retaining the target pixel, and filling the non-target pixel with other colors respectively, that is, changing pixel values of the other non-target pixels, and then obtaining a different filled image corresponding to an original image. Specifically, the identified target pixel needs to be retained first.
Step S302: and filling non-target pixels contained in each original image according to preset colors.
In a specific implementation, the non-target pixels are filled in a preset color on the basis of the original image. The preset color in this embodiment includes red, green, and blue, and the specific filling color may be adjusted accordingly according to the actual situation, which is not limited in this embodiment.
In a specific implementation, non-target pixels included in each original image are filled with the red, the green, and the blue, respectively.
Step S303: and obtaining a plurality of reference images corresponding to each original image according to the target pixels and the filled non-target pixels.
In a specific implementation, after the color filling is completed, in this embodiment, a plurality of filled images corresponding to each original image can be obtained according to the target pixel and the filled non-target pixels, and the original image
The number of fill images to which the image corresponds is consistent with the number of fill colors. Obtaining red filling reference images corresponding to the original images according to the target pixels and the red pixels; obtaining a green filling reference image corresponding to each original image according to the target pixel and the green pixel; and obtaining a blue filling reference image corresponding to each original image according to the target pixel and the blue pixel. E.g. red filled reference picture P 1 The reference image P is filled with green color and comprises target pixel and filled red pixel 2 Including a target pixel and a filled green pixel, and a blue filled reference image P 3 The method comprises the steps of filling a target pixel and a filled blue pixel, and adopting the method for each original imageSo that a red filled reference image, a green filled reference image and a blue filled reference image can be finally obtained
In a specific implementation, after the completion of the filling, a reference image set of each filling image may be constructed in the present embodiment. E.g. original image P 1 The corresponding red filled reference picture, green filled reference picture and blue filled reference picture are P 1R 、P 1G And P 1B Original image P 2 The corresponding red filled reference picture, green filled reference picture and blue filled reference picture are P 2R 、P 2G And P 2B Filling in the reference picture P according to red 1R And P 2R The reference image set corresponding to red filling can be obtained, and the reference image P is filled according to green 1G And P 2G A reference image set corresponding to the green filling can be obtained, and the reference image P is filled according to the blue 1B And P 2B A blue fill corresponding reference image set can be obtained.
The present embodiment retains the target pixels included in each original image; filling non-target pixels contained in each original image according to a preset color; obtaining a plurality of reference images corresponding to each original image according to the target pixels and the filled non-target pixels; and constructing a reference image set based on a plurality of reference images corresponding to each original image, and obtaining the reference image set which is more fit with the object to be reconstructed in a mode of reserving the target pixels and filling the non-target pixels.
Referring to fig. 7, fig. 7 is a schematic flowchart illustrating a three-dimensional model generating method according to a third embodiment of the present invention.
Based on the first embodiment, a third embodiment of the three-dimensional model generation method according to the present invention is provided.
In this embodiment, the step S50 specifically includes:
step S501: and determining a target voxel color according to the voxel colors.
In a specific implementation, in this embodiment, when the target three-dimensional model of the object to be reconstructed is generated according to the voxel colors and the voxel densities obtained by the respective reference image sets, the target voxel color needs to be determined according to the voxel colors obtained by the respective reference image sets. For example, c = (cr + cg + cb)/3, where c is the target voxel color, and cr, cg and cb are the voxel colors corresponding to the respective reference image sets, respectively.
Further, in this embodiment, corresponding target input vectors may be obtained based on different rendering requirements, where the target input vectors include a spatial point x position in the space where the object to be reconstructed is located and a view direction d of the original image or the reference image corresponding to the spatial point x, for example, (x, d), and are input into the nerve radiation field model for training, so as to obtain a voxel color and a voxel density corresponding to each reference image set, for example, a spatial coordinate position of a spatial point in the space where the object to be reconstructed is located shown in fig. 8 is x = (x, y, z) and a view direction d = (θ, Φ), and a multi-layered perceptron training is used to output (c, σ), where c is a color of the spatial point, includes three channel values of rgb, and σ is a voxel density of the position.
Step S502: and calculating the target voxel density according to the voxel density and the density correction coefficient.
In a specific implementation, in this embodiment, the target voxel density, that is, the voxel density corresponding to the target three-dimensional model, may also be calculated according to the voxel density and the density correction coefficient. For example, σ = (σ r + σ g + σ b)/3 × λ, where σ is the target voxel density, σ r, σ g, and σ b are the voxel densities corresponding to the respective reference image sets, and λ is the density correction coefficient.
Further, in this embodiment, the density correction factor may be calculated according to the voxel color, the target voxel color, and a preset constant. For example λ = K × exp [ -sqrt ((c-cr) 2 +(c-cg) 2 +(c-cb) 2 )/3/c]Wherein c is a target voxel color, cr, cg and cb are respectively voxel colors corresponding to the respective reference image sets, K is a preset constant, and belongs to an adjustable constant, and is used for balancing a final rendering effect, and a specific numerical value can be set correspondingly according to an actual rendering requirement, which is not limited in this embodiment, and it is emphasized that-sqrt ((c-cr) 2 +(c-cg) 2 +(c-cb) 2 For calculating the pixel variance.
It should be noted that the core of introducing the voxel density correction coefficient is to correct the voxel density according to the variance of voxel colors, and the voxel color variance corresponding to the target pixel is small, so that the voxel density corresponding to the target after multiplication by the voxel density coefficient is basically unchanged, but the voxel color variance corresponding to the non-target pixel is large, and the non-target voxel density after multiplication by the voxel density coefficient becomes a small value, so as to remove erroneous suspended voxels, and to fill the non-target pixels with different colors, the difference of the non-target pixels can be enlarged, which is beneficial to accurately identifying the non-target pixels, and is convenient for subsequent modeling.
Step S503: and generating a target three-dimensional model of the object to be reconstructed according to the target voxel color and the target voxel density.
In a specific implementation, the target three-dimensional model may be generated based on the target voxel color and the target voxel density obtained by the above calculation.
The embodiment determines a target voxel color according to the voxel color; calculating a target voxel density according to the voxel density and the density correction coefficient; and generating a target three-dimensional model of the object to be reconstructed according to the target voxel color and the target voxel density, correcting the voxel density according to the variance of the voxel color by introducing a voxel density correction coefficient, wherein the variance of the corresponding voxel color of the target pixel is small, so that the voxel density corresponding to the target multiplied by the voxel density coefficient is basically unchanged, but the color variance corresponding to the non-target pixel is large, and the non-target voxel density multiplied by the voxel density coefficient is changed into a smaller value, thereby removing the wrong suspended voxel.
Furthermore, an embodiment of the present invention further provides a storage medium, on which a three-dimensional model generation program is stored, and the three-dimensional model generation program implements the steps of the three-dimensional model generation method as described above when executed by a processor.
Since the storage medium adopts all technical solutions of all the embodiments, at least all the beneficial effects brought by the technical solutions of the embodiments are achieved, and no further description is given here.
Referring to fig. 9, fig. 9 is a block diagram showing a configuration of a three-dimensional model generating apparatus according to a first embodiment of the present invention.
As shown in fig. 9, a three-dimensional model generation apparatus according to an embodiment of the present invention includes:
an obtaining module 10, configured to obtain an input original image set including an image of an object to be reconstructed.
And an identifying module 20, configured to perform image segmentation on the original image set to obtain target pixels and non-target pixels included in each original image in the original image set.
And the processing module 30 is configured to process the target pixels and the non-target pixels, and obtain a plurality of reference image sets according to the processed pixels.
And the training module 40 is configured to train each reference image set to obtain a voxel color and a voxel density corresponding to each reference image set.
And the reconstruction module 50 is configured to generate a target three-dimensional model of the object to be reconstructed according to the voxel colors and the voxel density and density correction coefficients.
The method comprises the steps of obtaining an input original image set containing an object image to be reconstructed; performing image segmentation on the original image set to obtain target pixels and non-target pixels contained in each original image in the original image set; processing the target pixels and the non-target pixels, and obtaining a plurality of reference image sets according to the processed pixels; respectively training each reference image set to obtain a voxel color and a voxel density corresponding to each reference image set; the method comprises the steps of generating a target three-dimensional model of an object to be reconstructed according to a voxel color, a voxel density and a density correction coefficient, effectively distinguishing a target voxel from a non-target voxel based on the voxel color and the voxel density obtained by a nerve radiation field according to a reference model, introducing the voxel density correction coefficient, correcting the voxel density according to the voxel color variance, and enabling the corresponding voxel color variance of the target pixel to be small, so that the voxel density corresponding to a target after being multiplied by the voxel density coefficient is basically unchanged, but the voxel color variance corresponding to the non-target pixel is large, and the non-target voxel density after being multiplied by the voxel density coefficient can be changed into a small value, so that wrong suspended voxels are removed, and the rendering effect can be quickly optimized.
In an embodiment, the processing module 30 is further configured to reserve a target pixel included in each original image; filling non-target pixels contained in each original image according to a preset color; and obtaining a plurality of reference images corresponding to each original image according to the target pixels and the filled non-target pixels.
In one embodiment, the predetermined colors include red, green, and blue;
the processing module 30 is further configured to fill the non-target pixels included in each original image with the red, the green, and the blue colors, respectively.
In one embodiment, the filled non-target pixels are red pixels, green pixels and blue pixels, respectively;
the processing module 30 is further configured to obtain a red filling reference image corresponding to each original image according to the target pixel and the red pixel; obtaining a green filling reference image corresponding to each original image according to the target pixel and the green pixel; and obtaining a blue filling reference image corresponding to each original image according to the target pixel and the blue pixel.
In an embodiment, the reconstruction module 50 is further configured to determine a target voxel color according to the voxel color; calculating a target voxel density according to the voxel density and a density correction coefficient; and generating a target three-dimensional model of the object to be reconstructed according to the target voxel color and the target voxel density.
In an embodiment, the reconstruction module 50 is further configured to obtain a target input vector, where the target input vector includes a spatial point x position in a space where the object to be reconstructed is located and a view direction d of the original image or the reference image corresponding to the spatial point x; and determining the voxel color and the voxel density corresponding to each reference image set through the target input vector.
In an embodiment, the reconstruction module 50 is further configured to determine a density correction factor according to the voxel color, the target voxel color, and a preset constant.
It should be understood that the above is only an example, and the technical solution of the present invention is not limited in any way, and in a specific application, a person skilled in the art may set the technical solution as needed, and the present invention is not limited in this respect.
It should be noted that the above-described work flows are only exemplary, and do not limit the scope of the present invention, and in practical applications, a person skilled in the art may select some or all of them to achieve the purpose of the solution of the embodiment according to actual needs, and the present invention is not limited herein.
In addition, the technical details that are not described in detail in this embodiment may refer to the three-dimensional model generation method provided in any embodiment of the present invention, and are not described herein again.
Further, it is to be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or system. Without further limitation, an element defined by the phrases "comprising a," "8230," "8230," or "comprising" does not exclude the presence of other like elements in a process, method, article, or system comprising the element.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solution of the present invention or portions thereof that contribute to the prior art may be embodied in the form of a software product, where the computer software product is stored in a storage medium (e.g. Read Only Memory (ROM)/RAM, magnetic disk, optical disk), and includes several instructions for enabling a terminal device (e.g. a mobile phone, a computer, a server, or a network device) to execute the method according to the embodiments of the present invention.
The above description is only a preferred embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes, which are made by using the contents of the present specification and the accompanying drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.
Claims (10)
1. A three-dimensional model generation method, characterized by comprising:
acquiring an input original image set containing an object image to be reconstructed;
performing image segmentation on the original image set to obtain target pixels and non-target pixels contained in each original image in the original image set;
processing the target pixels and the non-target pixels, and obtaining a plurality of reference image sets according to the processed pixels;
training each reference image set respectively to obtain a voxel color and a voxel density corresponding to each reference image set;
and generating a target three-dimensional model of the object to be reconstructed according to the voxel colors and the voxel density and density correction coefficients.
2. The method of generating a three-dimensional model according to claim 1, wherein said processing said target pixels and said non-target pixels and deriving a plurality of reference image sets from the processed pixels comprises:
reserving target pixels contained in each original image;
filling non-target pixels contained in each original image according to a preset color;
and obtaining a plurality of reference images corresponding to each original image according to the target pixels and the filled non-target pixels.
3. The three-dimensional model generation method according to claim 2, wherein the preset colors include red, green, and blue;
the filling of the non-target pixels included in each original image according to the preset color includes:
filling non-target pixels included in each original image with the red, the green, and the blue, respectively.
4. The three-dimensional model generation method according to claim 3, wherein the filled non-target pixels are a red pixel, a green pixel, and a blue pixel, respectively;
the obtaining a plurality of reference images corresponding to each original image according to the target pixels and the filled non-target pixels comprises:
obtaining red filling reference images corresponding to the original images according to the target pixels and the red pixels;
obtaining a green filling reference image corresponding to each original image according to the target pixel and the green pixel;
and obtaining a blue filling reference image corresponding to each original image according to the target pixel and the blue pixel.
5. The method of generating a three-dimensional model of an object to be reconstructed according to the voxel colors and the voxel density and density correction factors of claim 1, comprising:
determining a target voxel color according to the voxel color;
calculating a target voxel density according to the voxel density and a density correction coefficient;
and generating a target three-dimensional model of the object to be reconstructed according to the target voxel color and the target voxel density.
6. The method of generating a three-dimensional model according to claim 5, wherein before determining a target voxel color from the voxel colors, further comprising:
acquiring a target input vector, wherein the target input vector comprises a space point x position in a space where the object to be reconstructed is located and a view direction d of the original image or the reference image corresponding to the space point x;
and determining the voxel color and the voxel density corresponding to each reference image set through the target input vector.
7. The method of generating a three-dimensional model according to claim 5, wherein before calculating the target voxel density from the voxel density and the density correction factor, the method further comprises:
and determining a density correction coefficient according to the voxel color, the target voxel color and a preset constant.
8. A three-dimensional model generation apparatus, characterized in that the three-dimensional model generation apparatus comprises:
the acquisition module is used for acquiring an input original image set containing an object image to be reconstructed;
the identification module is used for carrying out image segmentation on the original image set to obtain target pixels and non-target pixels contained in each original image in the original image set;
the processing module is used for processing the target pixels and the non-target pixels and obtaining a plurality of reference image sets according to the processed pixels;
the training module is used for respectively training each reference image set to obtain the voxel color and the voxel density corresponding to each reference image set;
and the reconstruction module is used for generating a target three-dimensional model of the object to be reconstructed according to the voxel colors and the voxel density and density correction coefficients.
9. A three-dimensional model generation device characterized by comprising: a memory, a processor, and a three-dimensional model generation program stored on the memory and run on the processor, the three-dimensional model generation program configured to implement the three-dimensional model generation method of any one of claims 1 to 7.
10. A storage medium characterized in that a three-dimensional model generation program is stored thereon, which when executed by a processor implements the three-dimensional model generation method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211127231.XA CN115457206A (en) | 2022-09-16 | 2022-09-16 | Three-dimensional model generation method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211127231.XA CN115457206A (en) | 2022-09-16 | 2022-09-16 | Three-dimensional model generation method, device, equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115457206A true CN115457206A (en) | 2022-12-09 |
Family
ID=84304612
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211127231.XA Pending CN115457206A (en) | 2022-09-16 | 2022-09-16 | Three-dimensional model generation method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115457206A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115797565A (en) * | 2022-12-20 | 2023-03-14 | 北京百度网讯科技有限公司 | Three-dimensional reconstruction model training method, three-dimensional reconstruction device and electronic equipment |
-
2022
- 2022-09-16 CN CN202211127231.XA patent/CN115457206A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115797565A (en) * | 2022-12-20 | 2023-03-14 | 北京百度网讯科技有限公司 | Three-dimensional reconstruction model training method, three-dimensional reconstruction device and electronic equipment |
| CN115797565B (en) * | 2022-12-20 | 2023-10-27 | 北京百度网讯科技有限公司 | Three-dimensional reconstruction model training method, three-dimensional reconstruction device and electronic equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111968235B (en) | Object attitude estimation method, device and system and computer equipment | |
| CN109003325B (en) | Three-dimensional reconstruction method, medium, device and computing equipment | |
| CN109697688B (en) | Method and device for image processing | |
| CN109859098B (en) | Face image fusion method and device, computer equipment and readable storage medium | |
| US6868191B2 (en) | System and method for median fusion of depth maps | |
| CN107484428B (en) | Method for displaying objects | |
| CN109636890B (en) | Texture fusion method and device, electronic equipment, storage medium and product | |
| US10169891B2 (en) | Producing three-dimensional representation based on images of a person | |
| CN111080776B (en) | Human body action three-dimensional data acquisition and reproduction processing method and system | |
| CN113689578A (en) | Human body data set generation method and device | |
| CN112669436B (en) | Deep learning sample generation method based on 3D point cloud | |
| CN113781621A (en) | Three-dimensional reconstruction processing method, device, equipment and storage medium | |
| CN115100383B (en) | Three-dimensional reconstruction method, device and equipment for mirror surface object based on common light source | |
| CN112348755A (en) | Image content restoration method, electronic device and storage medium | |
| CN114332125A (en) | Point cloud reconstruction method and device, electronic equipment and storage medium | |
| CN109448088A (en) | Method and device for rendering three-dimensional graphic wire frame, computer equipment and storage medium | |
| CN115457206A (en) | Three-dimensional model generation method, device, equipment and storage medium | |
| CN114972634A (en) | Multi-view three-dimensional deformable human face reconstruction method based on feature voxel fusion | |
| CN114820980A (en) | Three-dimensional reconstruction method and device, electronic equipment and readable storage medium | |
| JP5295044B2 (en) | Method and program for extracting mask image and method and program for constructing voxel data | |
| CN113989434A (en) | Human body three-dimensional reconstruction method and device | |
| CN110378948B (en) | 3D model reconstruction method and device and electronic equipment | |
| CN116188720A (en) | Digital person generation method, device, electronic equipment and storage medium | |
| CN111738061A (en) | Binocular vision stereo matching method based on regional feature extraction and storage medium | |
| CN113126944B (en) | Depth map display method, display device, electronic device, and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |