CN110517352A - A kind of three-dimensional rebuilding method of object, storage medium, terminal and system - Google Patents
A kind of three-dimensional rebuilding method of object, storage medium, terminal and system Download PDFInfo
- Publication number
- CN110517352A CN110517352A CN201910797141.3A CN201910797141A CN110517352A CN 110517352 A CN110517352 A CN 110517352A CN 201910797141 A CN201910797141 A CN 201910797141A CN 110517352 A CN110517352 A CN 110517352A
- Authority
- CN
- China
- Prior art keywords
- model
- fixed viewpoint
- image
- picture
- mask
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2200/00—Indexing scheme for image data processing or generation, in general
- G06T2200/08—Indexing scheme for image data processing or generation, in general involving all processing steps from image acquisition to 3D model generation
Abstract
The invention discloses a kind of three-dimensional rebuilding method of object, storage medium, terminal and systems, belong to picture reconstruction 3D modelling technique field, method includes: to extract the high dimensional feature of individual any angle picture, restores the first fixed viewpoint of object image according to high dimensional feature；Shape mask is generated according to the first fixed viewpoint image and then generates the 3D model of object.System includes U-shaped production confrontation network and 3D condition production confrontation network.High dimensional feature of the present invention by extracting single picture can reduce information disturbance to restore object fixed viewpoint i.e. the first fixed viewpoint view；According to fixed viewpoint view generation shape mask, be conducive to the efficiency, the accuracy that improve three-dimensional reconstruction, the feature true to nature suitable for visual angle picture, effect meets the needs of individual object picture real-time reconstruction 3D model of visual angle.
Description
Technical field
The present invention relates to single picture rebuild 3D modelling technique field more particularly to a kind of three-dimensional rebuilding method of object,
Storage medium, terminal and system.
Background technique
Three-dimensional reconstruction has a wide range of applications in computer vision and model field.In the past, researcher is usually to make
Three-dimensional reconstruction is solved with the picture of multiple different perspectivess, and realizing that three-dimensional reconstruction is from single picture is still one very tired
Difficult thing, because this needs very powerful model understandability to come from its shape information of the spatial prediction of a low-dimensional.
Recently, researcher carries out obtaining very big progress for the use of voxel prediction 3D is rebuild using CNN.Such methods are logical
Often considering the picture using a fixed viewpoint or a few visual angle, this is not particularly suited in actual application, because
Object can usually be observed by any angle in actual application, and the picture of shooting is also at any angle.But use random angle
The picture of degree carries out learning training and is difficult the effect obtained, because different perspectives picture bring otherness can upset network and mention
Its shape feature is taken, i.e. information disturbance is big.How to avoid different perspectives picture bring otherness from becoming and currently utilizes individual
Visual angle picture carries out object 3D Model Reconstruction urgent problem to be solved.
Summary of the invention
It is an object of the invention to overcome not realizing the three of object according to the picture of individual visual angle in the prior art
The deficiency rebuild is tieed up, three-dimensional rebuilding method, storage medium, terminal and the system of a kind of object are provided.
The purpose of the present invention is achieved through the following technical solutions, a kind of three-dimensional rebuilding method of object, method tool
Body is to extract the high dimensional feature of individual any angle picture and restore to obtain according to high dimensional feature including the use of fixed viewpoint image
First fixed viewpoint image of object generates shape mask according to the first fixed viewpoint image and then generates the 3D model of object.
Specifically, the first fixed viewpoint image generates shape mask and specifically includes: according to the first fixed viewpoint image zooming-out
The shape contour bianry image feature of object generates the shape mask in 3d space, calculation formula in turn specifically:
P_valid=P v=1 | mask=1 }=1
P_invalid=P v=1 | mask=0 }=0
Wherein, P represents the expectation that the shape contour bianry image corresponding position of model and object in the 3 d space has voxel,
P_valid indicates that the shape contour bianry image corresponding position of model and object has effective expectation of voxel, P_ in the 3 d space
Invalid invalid expectation of the shape contour bianry image corresponding position of model and object without voxel in the 3 d space, mask generation
Some pixel value in table shape contour bianry image, v indicate the voxel values of 3d space.
Specifically, the first fixed viewpoint includes side view visual angle, overlooks visual angle, front viewing angle.
Specifically, the first fixed viewpoint image for obtaining object is to fight network implementations by U-shaped production；According to
It is to fight network implementations by 3D condition production that one fixed viewpoint image, which generates shape mask and then generates the 3D model of object,
's.
Specifically, the arbiter in 3D condition production confrontation network joined and can set each other off with the first fixed viewpoint image
The shape mask penetrated, the shape mask can help arbiter to judge the true and false of 3D model and then adversely affect generator study wheel
Wide pictorial information.
Specifically, the first fixed viewpoint image step for obtaining object further includes trained production confrontation network:
Preprocessed data collection obtains the training dataset of several pictures of each angle of each object；
According to the generator and discriminator in training dataset alternately training production confrontation network model, generator is adjusted
With the weight of discriminator inner layers, and then obtain performance it is stable production confrontation network.
Specifically, before reduction obtains the first fixed viewpoint image step of object further include: by individual any angle picture
Carry out random areas cutting, reversion and color regularization.
The invention also includes a kind of storage mediums, are stored thereon with computer instruction, and computer instruction executes one when running
The step of three-dimensional rebuilding method of kind object.
The invention also includes a kind of terminal, including memory and processor, be stored on memory to transport on a processor
Capable computer instruction, which is characterized in that a kind of three-dimensional rebuilding method of object is executed when processor operation computer instruction
Step.
The invention also includes a kind of three-dimensional reconstruction system of object, system includes: for extracting individual any angle picture
High dimensional feature and being restored according to high dimensional feature obtain the U-shaped production confrontation network of the first fixed viewpoint image of object；With
Network is fought in the 3D condition production for the 3D model for generating object according to the first fixed viewpoint image.
Compared with prior art, the medicine have the advantages that
(1) present invention restores object fixed viewpoint (the first fixed viewpoint) view by extracting the high dimensional feature of single picture
Figure can reduce information disturbance, and fixed viewpoint (the first fixed viewpoint) the view generation shape mask obtained according to reduction, have
Conducive to efficiency, the accuracy for improving three-dimensional reconstruction, the 3D Model Reconstruction of object is finally completed according to the shape mask, and there is robust
Property the high, feature true to nature suitable for visual angle picture, effect, meet individual object picture real-time reconstruction 3D of visual angle
The needs of model.The present invention can generate corresponding object 3D by the object picture of simple one visual angle of input of user
Model.On the one hand, modeling personnel quickly can generate object 3D model by simple object picture, greatly reduce work
It measures, it is also possible to predict each visual angle figure of visual angle object picture.Alternatively, it is also possible to be applied to quick scene
Demonstration, under some simulated scenarios, the 3D model accuracy needed is not high, needs quickly to generate model object and scene,
Timely to be demonstrated.
(2) the U-shaped production confrontation network in the present invention is used to handle the reconstruction effect of different perspectives subject image generation
The big problem of difference, wherein trained U-shaped production confrontation network can predict the object of the subject image of visual angle
Fixed viewpoint (the first fixed viewpoint) view, to solve to interfere due to object illumination visual angle difference bring information；3D condition is raw
It is condition that an accepted way of doing sth, which fights Web vector graphic by fixed viewpoint (the first fixed viewpoint) view that U-shaped production confrontation network generates, raw
At corresponding 3D model.
Detailed description of the invention
Specific embodiments of the present invention will be described in further detail with reference to the accompanying drawing, attached drawing described herein
For providing further understanding of the present application, constitute part of this application, in the drawings using identical with reference to mark
Number indicate the same or similar part, illustrative embodiments of the present application and the description thereof are used to explain the present application, do not constitute
Improper restriction to the application.In figure:
Fig. 1 is the method flow diagram of the embodiment of the present invention 1；
Fig. 2 is the 3D model schematic generated by single picture；
Fig. 3 is the effect picture of the method for the present invention；
Fig. 4 is 2 system framework schematic diagram of the embodiment of the present invention；
Fig. 5 is that the 3D condition production of shape mask fights schematic network structure；
Fig. 6 is that the U-shaped production of the present invention fights network structure.
Specific embodiment
Technical solution of the present invention is clearly and completely described with reference to the accompanying drawing, it is clear that described embodiment
It is a part of the embodiment of the present invention, instead of all the embodiments.Based on the embodiments of the present invention, ordinary skill people
Member's every other embodiment obtained without making creative work, shall fall within the protection scope of the present invention.
In the description of the present invention, it should be noted that belong to "center", "upper", "lower", "left", "right", "vertical",
The direction of the instructions such as "horizontal", "inner", "outside" or positional relationship are to be merely for convenience of retouching based on attached drawing direction or positional relationship
It states the present invention and simplifies description, rather than the device or element of indication or suggestion meaning must have a particular orientation, with specific
Orientation construction and operation, therefore be not considered as limiting the invention.In addition, belonging to " first ", " second " is only used for retouching
Purpose is stated, relative importance is not understood to indicate or imply.
In the description of the present invention, it should be noted that unless otherwise clearly defined and limited, belong to " installation ", " phase
Even ", " connection " shall be understood in a broad sense, for example, it may be being fixedly connected, may be a detachable connection, or be integrally connected；It can
To be mechanical connection, it is also possible to be electrically connected；It can be directly connected, can also can be indirectly connected through an intermediary
Connection inside two elements.For the ordinary skill in the art, above-mentioned term can be understood at this with concrete condition
Concrete meaning in invention.
As long as in addition, the non-structure each other of technical characteristic involved in invention described below different embodiments
It can be combined with each other at conflict.
Embodiment 1
As shown in Figure 1, in embodiment 1, a kind of three-dimensional rebuilding method of object, specifically includes the following steps:
S01: training network model；It include that U-shaped production confrontation network and 3D condition are raw in the present embodiment production network
An accepted way of doing sth fights network, and U-shaped production confrontation network and 3D condition production confrontation network are all based on production confrontation network, when
Network is fought different from original production.Wherein U-shaped production confrontation network in input generator G be not one with
Machine vector, but the object picture of a visual angle.Input generator G's is in 3D condition production confrontation network of network
The profile side view of corresponding object.
Further, training whole network specifically includes following sub-step:
S011: data set is pre-processed；What it is based on public data collection ShapeNet offer includes 20 class model data,
On this data set, irradiating angle is randomly provided to each object, the picture of 10 various angles is generated, as U-shaped production
Fight the training dataset of network.Since the data set ShapeNet model provided is mesh model, for the 3D of mesh model
Model, it would be desirable to the voxel model of 64*64*64 space size is translated into, as 3D condition production confrontation network
Training dataset.
S012: alternately training generator G and discriminator D, the weight of generator and discriminator inner layers is adjusted, and then obtain
The generation network stable to performance；Specifically, the image input generator G for the data set that pretreatment obtains is predicted to obtain
It exports image G (X), and this figure G (X) is sent to discriminator D with original true picture X respectively and is distinguished the true from the false, by the result
Instruct training generator G and discriminator D.More specifically, training discriminator D when, it is desirable that generator G generate image and by its
It is output to discriminator D；According to input/target image to (X, Y) and input/output image to (X, G (X)), discriminator D identification by
The image that generator G is provided is the probability of true picture；Discriminator D according to input/target image to and input/output image pair
Error in classification adjustment discriminator D inner layers weight, specific formula are as follows:
VCGAN(G, D)=E(X,Y)[logD(X,Y)]+
EX[log(1-D(X,G(X)))]
In training generator G, acquired according to the differentiation of discriminator D result, that is, error in classification and being calculated from following formula
Output image and target image between difference adjust the weight of each layer of generator G interior:
Further, the stochastic gradient descent for the use of batch size being 8, and Adam optimizer is used, and use Batch
Standardization.The present embodiment replaces between the gradient updating of generator G and the gradient updating of discriminator D.It changes when having trained 80
Generation, network performance are stablized.
S02: data prediction；Specifically, the object target picture of arbitrary viewing angles is cut into the picture of fixed size, and
Background area color is removed, is filled with green, one whole image to be processed is formed, the generation network as processing visual angle
Input.
S03: extracting the high dimensional feature of individual any angle picture, restores the first fixed viewpoint of object figure according to high dimensional feature
Picture；Wherein, the first fixed viewpoint includes but is not limited to side view visual angle, and front viewing angle overlooks visual angle etc., as an option, this reality
The first fixed viewpoint for applying example is specially to test visual angle.
Further, the high dimensional feature for extracting individual any angle image after pre-processing restores any according to high dimensional feature
The side view figure image of visual angle subject image, extracts shape contour bianry image according to side view object boundary.
Further, processing visual angle figure be the U-shaped production confrontation network of fixed viewpoint figure include generator G with
Discriminator D；Wherein generator G is formed (X, Y) training by feature/true picture, and wherein X is the arbitrary viewing angles figure of jobbie
Piece, Y are the side views of object corresponding with X；Trained generator G converts the X of input, with obtain for object
Body-side view G (X)；Trained discriminator D is used to differentiate whether unknown images to be the image G (X) generated by generator, not
Know that image includes real goal image Y from the data set or output image G (X) for carrying out self-generator G；
Generate the objective function of network are as follows:
Wherein:
VCGAN(G, D)=E(X,Y)[logD(X,Y)]+
EX[log(1-D(X,G(X)))]；
In formula, D (X, Y) and D (X, G (X)) are that discriminator D is judged as true as a result, representing to the differentiation of different images pair
Probability；And E(X,Y)Expression adds up to all feature/true pictures from sample to the differentiation calculated result of (X, Y),
And the expectation form for further using probability distribution is write out；EXThen indicate to be to carry out (X, G (X)) feature/generation image
State respective handling；
In the training process, the target of generator G is just to try to generate true picture and go to cheat to differentiate discriminator D.And
The target of discriminator D is just to try to the generator G picture generated and true picture to be distinguished from.In this way, generator G and mirror
Other device D constitutes a dynamic minmax game, and under optimal state, generator G, which can be generated, to be enough " with false random
Picture G (X) very ".For identifying for D, it is difficult to determine whether true the picture that generator G is generated is actually, therefore D
(G (X))=0.5.
Further, U-shaped production confrontation individual visual angle picture of network processes is fixed viewpoint (side view visual angle) figure
In the step of piece, in order to bring more changeabilities to training data, random areas cutting, overturning, color are carried out to input data
Color regularization.
S04: generating shape mask according to the first fixed viewpoint image and then generates the 3D model of object.Specifically, it generates
It includes 3D generator G that the 3D confrontation of model, which generates network,1With 3D discriminator D1；
Further, the step of generating shape mask Mask information is as follows:
S041: it will enter into 3D generator G12D contour of object figure based on generate one and cover the model voxel
Shape mask Mask.Its calculation formula is:
P_valid=P v=1 | mask=1 }=1
P_invalid=P v=1 | mask=0 }=0
Wherein, P represents the expectation that the shape contour bianry image corresponding position of model and object in the 3 d space has voxel,
P_valid indicates that the shape contour bianry image corresponding position of model and object has effective expectation of voxel, P_ in the 3 d space
Invalid invalid expectation of the shape contour bianry image corresponding position of model and object without voxel in the 3 d space, mask generation
Some pixel value in table shape contour bianry image, v indicate the voxel values of 3d space.In 2D contour of object figure, if
The pixel value of some position be 1 and pixel (x, y)=1, then in 3D voxel space we will be arranged voxels (from
(x, y, 0) arrives (x, y, 63)) it is 1.As shown in figure 4, Mask (y) is by generating one for the model based on 2D contour of object figure
The shape mask Mask that voxel covers.
S042: the 3D model of object is generated according to shape mask；3D generator G1By feature/true picture to (X1,Y1) instruction
Practice and formed, wherein X1It is the profile side view of pending object, object area is filled black, and profile periphery is filled out by white
It fills, Y1It is and X1The 3D voxel model of corresponding object, in three dimensions, object overlay area is by 1 filling, remaining region is by 0
Filling；Trained 3D generator G1To the X of input1It carries out feature extraction and 3D model generates, to obtain the voxel rebuild
Model G (X1)；Trained 3D discriminator D is for differentiating whether Unknown Model is the model G (X generated by 3D generator1),
Unknown Model includes the real goal model Y from data set1Or come from 3D generator G1Output model G (X1)。
Further, 3D confrontation generates the objective function of network are as follows:
In above formula, D (x | Mask (y)) and D (G (z | y) | Mask (y)) it is 3D discriminator D1Different models pair are sentenced
Really rate is not judged as it as a result, representing；And EX~PdataWith EZ~PnoiseIt indicates respectively to carrying out really all figures from sample and generation
Picture is with model to (X1,Y1) differentiation calculated result add up, and the expectation form for further using probability distribution is write out.
S05: theme rendering；Specifically, after generating 3D model, be based on unreal rendering engine, carry out the rendering of model with
It shows.In order to realize the modeling rendering of output model and show, the present embodiment has built one based on unreal rendering engine
Model rendering device, the voxel model obtained after obtaining a upper section, rendering engine can generate small according to the position of voxel model
Cube, each cube represent corresponding position and belong to model coverage areas, establish 3D voxel model with this.In addition, establishing
Illumination relationship, as shown in Fig. 2, rendering result of the voxel model in renderer.
S7: terminal applies communication；Client requests server-side processing by http protocol, is returned the result.For reality
Now more easily application, using being divided into server-side and client.Client requests the service of server-side by http protocol.Its
Middle client is mainly responsible for processing user's interaction response, and UI is shown, 3D model rendering function.Server-side runs main network
It is that U-shaped confrontation type generates network and 3D condition production confrontation network respectively, is responsible for processing core computing function, including to client
The response at end pre-processes original image, generates profile side view, generates object model, returns result to client.
The present embodiment additionally provides a kind of storage medium, is stored thereon with computer instruction, and computer instruction is held when running
In row embodiment 1 the step of a kind of three-dimensional rebuilding method of object.
Based on this understanding, the technical solution of the present embodiment substantially the part that contributes to existing technology in other words
Or the part of the technical solution can be embodied in the form of software products, which is stored in one and deposits
In storage media, including some instructions are used so that a computer equipment (can be personal computer, server or network
Equipment etc.) execute all or part of the steps of each embodiment method of the present invention.And storage medium above-mentioned includes: USB flash disk, movement
Hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory,
RAM), the various media that can store program code such as magnetic or disk.
The present embodiment also provides a kind of terminal, including memory and processor, and being stored on memory can be on a processor
The computer instruction of operation, processor execute a kind of three-dimensional rebuilding method of object in embodiment 1 when running computer instruction
Step.Processor can be monokaryon or multicore central processing unit perhaps specific integrated circuit or be configured to implement this
One or more integrated circuit of invention.
Each functional unit in embodiment provided by the invention can integrate in one processing unit, be also possible to each
A unit physically exists alone, and can also be integrated in one unit with two or more units.
The present embodiment further includes a kind of equipment, what the three-dimensional rebuilding method which is used to receive a kind of above-mentioned object obtained
3D model, is shown or other purposes.
The present invention by extract single picture high dimensional feature to restore object fixed viewpoint (the first fixed viewpoint) view,
It can reduce information disturbance, further according to fixed viewpoint (the first fixed viewpoint) view generation shape mask, be conducive to improve three-dimensional
The efficiency of reconstruction, accuracy, and have the characteristics that robustness is high, true to nature suitable for visual angle picture, effect, it meets any
The needs of individual object picture real-time reconstruction 3D model at visual angle, as shown in figure 3, the horizontally-arranged effect picture of third is using this in figure
Method generate object 3D model, and second it is horizontally-arranged be not using this method (not restoring object fixed viewpoint) generate object
3D illustraton of model.The present invention can generate corresponding object 3D by the object picture of simple one visual angle of input of user
Model.On the one hand, modeling personnel quickly can generate object 3D model by simple object picture, greatly reduce work
It measures, it is also possible to predict each visual angle figure of visual angle object picture.Alternatively, it is also possible to be applied to quick scene
Demonstration, under some simulated scenarios, the 3D model accuracy needed is not high, needs quickly to generate model object and scene,
Timely to be demonstrated.
Embodiment 2
The present embodiment and embodiment 1 are based on identical inventive concept, provide a kind of object on the basis of embodiment 1
Three-dimensional reconstruction system, as shown in figure 4, the system specifically includes U-shaped production confrontation network and 3D condition production confrontation network,
U-shaped production confrontation network extracts the high-order feature of individual any angle picture to restore object the first fixed viewpoint image and object
Body the first fixed viewpoint image is input to 3D condition production confrontation network；It is fixed according to first that 3D condition production fights network
The shape contour bianry image feature of multi-view image generates shape mask and then generates the 3D model of object.U-shaped production confrontation
Network and 3D condition production confrontation network include generator and discriminator, and Fig. 5 is the 3D condition production pair of shape mask
Anti- schematic network structure, Fig. 6 are that the U-shaped production of the present invention fights network structure.
Further, the architecture of generator G is coder-decoder network, and encoder section is by a series of complete
Full convolutional layer (convolution size is 3 × 3) and resolution ratio reduces and forms, and decoder by it is a series of deconvolute/up-sample form.
In addition, in decoded portion, every layer of layer for being therefore connected to low resolution and it is additional skip connection be connected to
The encoder layer of its equal resolution (U-net).These additional connections allow by directly passing the low-level information from input
The defeated bottleneck that coder-decoder is bypassed to output.
Further, generator G includes the m layer coder and m layer decoder being linked in sequence, the input terminal input of encoder
The output end of image X, decoder export image G (X)；Wherein, each encoder includes the convolutional layer being linked in sequence, Batch
Norm layers and ReLU layers, each decoder includes Norm layers and ReLU layers of deconvolution/up-sampling layer, Batch；And n-th layer
The jump connection of the input terminal of the output end of convolutional layer and m-n layers of warp lamination, wherein m is the number of plies.
Further, encoder gradually decreases the Spatial Dimension of pond layer, and decoder gradually repairs the details and sky of object
Between dimension.Usually there is quick connection between encoder and decoder, therefore decoder can be helped preferably to repair the thin of target
Section, extracts the high dimensional feature of image.Because many information are shared with outlet chamber in input in network, then needing will be in encoder
Information be directly passed to decoder.In order to realize that information is contributed, the jump between n-th layer and m-n layers is increased in network
Connection.Wherein m is the network number of plies, i.e., n-th layer (encoder) information is directly passed to m-n layers of (decoding by each jump connection
Device).
Further, discriminator D includes the multiple convolutional layers being linked in sequence, and includes Batch Norm between adjacent convolutional layer
Layer and ReLU layers.
Further, the parameters weighting comprising several training optimizations in every layer network in generator G and discriminator D,
Its value is updated by training dynamic.
Further, in the present embodiment, the arbitrary viewing angles picture X size of jobbie is 512 × 512 × 3, wherein
Input channel is 3, and expression input picture is RGB triple channel, and 512 × 512 representative image resolution ratio are 512 × 512 pixels；Output
Image G (X) size is 512 × 512 × 3, and wherein input channel is 3, and expression input picture is RGB triple channel；It wherein can also be with
Use 256 × 256 × 3 image in different resolution.The image size that each layer coder obtains is respectively as follows: 256 × 256 × 64,128 ×
128 × 128,64 × 64 × 256,32 × 32 × 512,16 × 16 × 512,8 × 8 × 512,4 × 4 × 512,2 × 2 × 512, it compiles
The characteristics of image size of the output end output of code device is 1 × 1 × 1024；The image size that each layer decoder obtains is respectively 2
×2×512、4×4×512、8×8×512、16×16×512、32×32×512、64×64×256、128×128×
128、256×256×64、512×512×3。
The present embodiment further includes a kind of equipment, using the three-dimensional reconstruction system realization for individual visual angle object picture
The three-dimensional reconstruction of picture, for purposes such as model displays.
U-shaped production confrontation network in the present invention is used to handle the reconstruction difference on effect of different perspectives subject image generation
Big problem, wherein trained U-shaped production confrontation network can predict that the object of the subject image of visual angle is fixed
Visual angle (the first fixed viewpoint) view, to solve to interfere due to object illumination visual angle difference bring information；3D condition production
It is condition, generation pair that Web vector graphic, which is fought, by fixed viewpoint (the first fixed viewpoint) view that U-shaped production confrontation network generates
The 3D model answered.
The above specific embodiment is detailed description of the invention, and it cannot be said that a specific embodiment of the invention office
It is limited to these explanations, for those of ordinary skill in the art to which the present invention belongs, before not departing from present inventive concept
It puts, several simple deductions and substitution can also be made, all shall be regarded as belonging to protection scope of the present invention.
Claims (10)
1. a kind of three-dimensional rebuilding method of object, it is characterised in that: the described method includes:
It is to extract the high dimensional feature of individual any angle picture and restored according to the high dimensional feature using fixed viewpoint image
To the first fixed viewpoint image of object, shape mask is generated according to the first fixed viewpoint image and then generates the 3D of object
Model.
2. a kind of three-dimensional rebuilding method of object according to claim 1, it is characterised in that: according to the described first fixed view
Angle image generates shape mask and specifically includes: according to the shape contour bianry image of the first fixed viewpoint image zooming-out object
Shape mask in feature and then generation 3d space, shape mask calculation formula specifically:
P_valid=P v=1 | mask=1 }=1
P_invalid=P v=1 | mask=0 }=0
Wherein, P represents the expectation that the shape contour bianry image corresponding position of model and object in the 3 d space has voxel, P_
Valid indicates that the shape contour bianry image corresponding position of model and object has effective expectation of voxel, P_ in the 3 d space
Invalid invalid expectation of the shape contour bianry image corresponding position of model and object without voxel in the 3 d space, mask generation
Some pixel value in table shape contour bianry image, v indicate the voxel values of 3d space.
3. a kind of three-dimensional rebuilding method of object according to claim 1, it is characterised in that: the first fixed viewpoint packet
It includes side view visual angle, overlook visual angle, front viewing angle.
4. a kind of three-dimensional rebuilding method of object according to claim 1, it is characterised in that: described to obtain the first of object
Fixed viewpoint image is to fight network implementations by U-shaped production；
Generating shape mask according to the first fixed viewpoint image and then generating the 3D model of object is generated by 3D condition
Formula fights network implementations.
5. a kind of three-dimensional rebuilding method of object according to claim 4, it is characterised in that: the 3D condition production pair
Arbiter in anti-network joined can be with the shape mask of the first fixed viewpoint image phase mapping.
6. a kind of three-dimensional rebuilding method of object according to claim 4, it is characterised in that: described to obtain the first of object
Fixed viewpoint image step further includes trained production confrontation network:
Preprocessed data collection obtains the training dataset of several pictures of each angle of each object；
According to the generator and discriminator in training dataset alternately the training production confrontation network model, generator is adjusted
With the weight of discriminator inner layers, and then obtain performance it is stable production confrontation network.
7. a kind of three-dimensional rebuilding method of object according to claim 4, it is characterised in that: the reduction obtains object
Before first fixed viewpoint image step further include:
Individual any angle picture is subjected to random areas cutting, reversion and color regularization.
8. a kind of storage medium, is stored thereon with computer instruction, it is characterised in that: the right of execution when computer instruction is run
Benefit requires a kind of the step of three-dimensional rebuilding method of object described in 1-7 any one.
9. a kind of terminal, including memory and processor, the meter that can be run on the processor is stored on the memory
Calculation machine instruction, which is characterized in that perform claim requires described in 1-7 any one when the processor runs the computer instruction
A kind of object three-dimensional rebuilding method the step of.
10. a kind of three-dimensional reconstruction system of object, it is characterised in that: the system comprises:
Consolidate for extracting the high dimensional feature of individual any angle picture and restoring to obtain the first of object according to the high dimensional feature
Determine the U-shaped production confrontation network of multi-view image；
3D condition production for generating the 3D model of object according to the first fixed viewpoint image fights network.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910797141.3A CN110517352B (en) | 2019-08-27 | 2019-08-27 | Three-dimensional reconstruction method, storage medium, terminal and system of object |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910797141.3A CN110517352B (en) | 2019-08-27 | 2019-08-27 | Three-dimensional reconstruction method, storage medium, terminal and system of object |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110517352A true CN110517352A (en) | 2019-11-29 |
| CN110517352B CN110517352B (en) | 2022-06-03 |
Family
ID=68628192
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910797141.3A Active CN110517352B (en) | 2019-08-27 | 2019-08-27 | Three-dimensional reconstruction method, storage medium, terminal and system of object |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110517352B (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111583398A (en) * | 2020-05-15 | 2020-08-25 | 网易（杭州）网络有限公司 | Image display method and device, electronic equipment and computer readable storage medium |
| CN112070893A (en) * | 2020-09-15 | 2020-12-11 | 大连理工大学 | Dynamic sea surface three-dimensional modeling method based on deep learning and storage medium |
| CN112884669A (en) * | 2021-02-25 | 2021-06-01 | 电子科技大学 | Image restoration method based on multi-scale content attention mechanism, storage medium and terminal |
| CN114255313A (en) * | 2022-02-28 | 2022-03-29 | 深圳星坊科技有限公司 | Three-dimensional reconstruction method and device for mirror surface object, computer equipment and storage medium |
| CN117456144A (en) * | 2023-11-10 | 2024-01-26 | 中国人民解放军海军航空大学 | Target building three-dimensional model optimization method based on visible light remote sensing image |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108805977A (en) * | 2018-06-06 | 2018-11-13 | 浙江大学 | A kind of face three-dimensional rebuilding method based on end-to-end convolutional neural networks |
| US20190051048A1 (en) * | 2016-04-19 | 2019-02-14 | Zhejiang University | Method for single-image-based fully automatic three-dimensional hair modeling |
| US20190139179A1 (en) * | 2017-11-03 | 2019-05-09 | Baidu Usa Llc | Systems and methods for unsupervised learning of geometry from images using depth-normal consistency |
| CN109920054A (en) * | 2019-03-29 | 2019-06-21 | 电子科技大学 | A kind of adjustable 3D object generation method generating confrontation network based on three-dimensional boundaries frame |
| CN109977922A (en) * | 2019-04-11 | 2019-07-05 | 电子科技大学 | A kind of pedestrian's mask generation method based on generation confrontation network |
| CN109993825A (en) * | 2019-03-11 | 2019-07-09 | 北京工业大学 | A kind of three-dimensional rebuilding method based on deep learning |
| CN110047128A (en) * | 2018-01-15 | 2019-07-23 | 西门子保健有限责任公司 | The method and system of X ray CT volume and segmentation mask is rebuild from several X-ray radiogram 3D |
| CN110084845A (en) * | 2019-04-30 | 2019-08-02 | 王智华 | Deformation Prediction method, apparatus and computer readable storage medium |
| CN110136063A (en) * | 2019-05-13 | 2019-08-16 | 南京信息工程大学 | A kind of single image super resolution ratio reconstruction method generating confrontation network based on condition |
-
2019
- 2019-08-27 CN CN201910797141.3A patent/CN110517352B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20190051048A1 (en) * | 2016-04-19 | 2019-02-14 | Zhejiang University | Method for single-image-based fully automatic three-dimensional hair modeling |
| US20190139179A1 (en) * | 2017-11-03 | 2019-05-09 | Baidu Usa Llc | Systems and methods for unsupervised learning of geometry from images using depth-normal consistency |
| CN110047128A (en) * | 2018-01-15 | 2019-07-23 | 西门子保健有限责任公司 | The method and system of X ray CT volume and segmentation mask is rebuild from several X-ray radiogram 3D |
| CN108805977A (en) * | 2018-06-06 | 2018-11-13 | 浙江大学 | A kind of face three-dimensional rebuilding method based on end-to-end convolutional neural networks |
| CN109993825A (en) * | 2019-03-11 | 2019-07-09 | 北京工业大学 | A kind of three-dimensional rebuilding method based on deep learning |
| CN109920054A (en) * | 2019-03-29 | 2019-06-21 | 电子科技大学 | A kind of adjustable 3D object generation method generating confrontation network based on three-dimensional boundaries frame |
| CN109977922A (en) * | 2019-04-11 | 2019-07-05 | 电子科技大学 | A kind of pedestrian's mask generation method based on generation confrontation network |
| CN110084845A (en) * | 2019-04-30 | 2019-08-02 | 王智华 | Deformation Prediction method, apparatus and computer readable storage medium |
| CN110136063A (en) * | 2019-05-13 | 2019-08-16 | 南京信息工程大学 | A kind of single image super resolution ratio reconstruction method generating confrontation network based on condition |
Non-Patent Citations (4)
| Title |
|---|
| BO YANG等: ""3D Object Reconstruction from a Single Depth View with Adversarial Learning"", 《2017 IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION WORKSHOPS》 * |
| PING KUANG等: ""3D Bounding Box Generative Adversarial Nets"", 《IN PROCEEDINGS OF THE INTERNATIONAL CONFERENCE ON ADVANCES IN COMPUTER TECHNOLOGY, INFORMATION SCIENCE AND COMMUNICATIONS (CTISC 2019)》 * |
| PING KUANG等: ""Masked 3D conditional generative adversarial network for rock mesh generation"", 《CLUSTER COMPUTING》 * |
| 陈加等: ""深度学习在基于单幅图像的物体三维重建中的应用"", 《自动化学报》 * |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111583398A (en) * | 2020-05-15 | 2020-08-25 | 网易（杭州）网络有限公司 | Image display method and device, electronic equipment and computer readable storage medium |
| CN111583398B (en) * | 2020-05-15 | 2023-06-13 | 网易（杭州）网络有限公司 | Image display method, device, electronic equipment and computer readable storage medium |
| CN112070893A (en) * | 2020-09-15 | 2020-12-11 | 大连理工大学 | Dynamic sea surface three-dimensional modeling method based on deep learning and storage medium |
| CN112070893B (en) * | 2020-09-15 | 2024-04-02 | 大连理工大学 | Dynamic sea surface three-dimensional modeling method based on deep learning and storage medium |
| CN112884669A (en) * | 2021-02-25 | 2021-06-01 | 电子科技大学 | Image restoration method based on multi-scale content attention mechanism, storage medium and terminal |
| CN114255313A (en) * | 2022-02-28 | 2022-03-29 | 深圳星坊科技有限公司 | Three-dimensional reconstruction method and device for mirror surface object, computer equipment and storage medium |
| CN117456144A (en) * | 2023-11-10 | 2024-01-26 | 中国人民解放军海军航空大学 | Target building three-dimensional model optimization method based on visible light remote sensing image |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110517352B (en) | 2022-06-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110517352A (en) | A kind of three-dimensional rebuilding method of object, storage medium, terminal and system | |
| CN111986307B (en) | 3D object reconstruction using a light grid representation | |
| Gu et al. | Nerfdiff: Single-image view synthesis with nerf-guided distillation from 3d-aware diffusion | |
| CN111369681B (en) | Three-dimensional model reconstruction method, device, equipment and storage medium | |
| Kaufman et al. | Overview of volume rendering. | |
| US7940279B2 (en) | System and method for rendering of texel imagery | |
| CN110390638A (en) | A kind of high-resolution three-dimension voxel model method for reconstructing | |
| CN105453139A (en) | Sparse GPU voxelization for 3D surface reconstruction | |
| US10713844B2 (en) | Rendering based generation of occlusion culling models | |
| CN115100339A (en) | Image generation method and device, electronic equipment and storage medium | |
| CN109936745A (en) | For improving the method and system of the decompression of original video data | |
| CN113313828B (en) | Three-dimensional reconstruction method and system based on single-picture intrinsic image decomposition | |
| CN106408626A (en) | Graphics processing systems | |
| CN110490807A (en) | Image rebuilding method, device and storage medium | |
| CN115797561A (en) | Three-dimensional reconstruction method, device and readable storage medium | |
| CN115375828A (en) | Model shadow generation method, device, equipment and medium | |
| CN116912148B (en) | Image enhancement method, device, computer equipment and computer readable storage medium | |
| CN114429531A (en) | Virtual viewpoint image generation method and device | |
| US11908067B1 (en) | Method and device for gigapixel-level light field intelligent reconstruction of large-scale scene | |
| CN117315153A (en) | Human body reconstruction and rendering method and device for cooperative light field and occupied field | |
| CN115908753B (en) | Method and related device for reconstructing whole-body human body grid surface | |
| CN115908205A (en) | Image restoration method and device, electronic equipment and storage medium | |
| CN115222917A (en) | Training method, device and equipment for three-dimensional reconstruction model and storage medium | |
| EP2266101B1 (en) | Method and device for contact simulation using layered depth images | |
| CN115375836A (en) | Point cloud fusion three-dimensional reconstruction method and system based on multivariate confidence filtering |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |