CN109300188A - Threedimensional model processing method and processing device - Google Patents
Threedimensional model processing method and processing device Download PDFInfo
- Publication number
- CN109300188A CN109300188A CN201811236759.4A CN201811236759A CN109300188A CN 109300188 A CN109300188 A CN 109300188A CN 201811236759 A CN201811236759 A CN 201811236759A CN 109300188 A CN109300188 A CN 109300188A
- Authority
- CN
- China
- Prior art keywords
- original
- human face
- dimensional model
- face
- disparity
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 22
- 238000012545 processing Methods 0.000 title claims abstract description 17
- 238000000034 method Methods 0.000 claims abstract description 47
- 230000009466 transformation Effects 0.000 claims abstract description 24
- 230000000875 corresponding effect Effects 0.000 claims description 49
- 239000011159 matrix material Substances 0.000 claims description 31
- 238000004590 computer program Methods 0.000 claims description 11
- 238000006073 displacement reaction Methods 0.000 claims description 8
- 238000012935 Averaging Methods 0.000 claims description 3
- 230000002596 correlated effect Effects 0.000 claims description 3
- 238000012805 post-processing Methods 0.000 abstract description 8
- 238000005457 optimization Methods 0.000 abstract description 3
- 210000000887 face Anatomy 0.000 description 24
- 230000006870 function Effects 0.000 description 12
- 238000010586 diagram Methods 0.000 description 9
- 238000013519 translation Methods 0.000 description 9
- 230000008569 process Effects 0.000 description 6
- 238000004422 calculation algorithm Methods 0.000 description 4
- 241000208340 Araliaceae Species 0.000 description 3
- 235000005035 Panax pseudoginseng ssp. pseudoginseng Nutrition 0.000 description 3
- 235000003140 Panax quinquefolius Nutrition 0.000 description 3
- 235000008434 ginseng Nutrition 0.000 description 3
- 230000007246 mechanism Effects 0.000 description 3
- 230000008439 repair process Effects 0.000 description 3
- 230000002159 abnormal effect Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 230000007423 decrease Effects 0.000 description 2
- 230000004069 differentiation Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 210000003128 head Anatomy 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 241001269238 Data Species 0.000 description 1
- 241000252794 Sphinx Species 0.000 description 1
- 230000000712 assembly Effects 0.000 description 1
- 238000000429 assembly Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000015572 biosynthetic process Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 235000013399 edible fruits Nutrition 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 238000005498 polishing Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000007634 remodeling Methods 0.000 description 1
- 239000000126 substance Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
Landscapes
- Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Computer Graphics (AREA)
- Geometry (AREA)
- Software Systems (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Processing Or Creating Images (AREA)
Abstract
The present invention relates to dimensional Modeling Technology fields, provide a kind of threedimensional model processing method and processing device.Wherein, threedimensional model processing method, comprising: obtain the original human face three-dimensional model of face to be modeled, original human face three-dimensional model includes at least one original vertices；Calculate the disparity between original vertices and corresponding standard peak in standard faces threedimensional model；Geometric transformation is carried out to original vertices to be treated based on disparity, is obtained using the position of standard peak as the representative points of reference；Original vertices to be treated are replaced with into corresponding representative points in original human face three-dimensional model, obtain transformed human face three-dimensional model.The above method can be used for carrying out post-processing optimization to original human face three-dimensional model, and treated, and original human face three-dimensional model quality is higher, can be applied in various application scenarios, improve user experience.
Description
Technical field
The present invention relates to dimensional Modeling Technology fields, in particular to a kind of threedimensional model processing method and processing device.
Background technique
Currently, in the unlock of such as three-dimension virtual reality, face, face remodeling, three-dimensional U.S. face, three-dimensional polishing, three-dimensional fitting
In applying, it can all be related to face three-dimensional modeling.The human face three-dimensional model matter that currently used such as the methods of structure light is established
It measures bad, is easy to appear the situation of mould shapes exception, seriously affects the user experience of above-mentioned application.
Summary of the invention
In view of this, the embodiment of the present invention provides a kind of threedimensional model processing method and processing device, with established face three
Dimension module is post-processed, to improve its quality.
To achieve the above object, the invention provides the following technical scheme:
In a first aspect, the embodiment of the present invention provides a kind of threedimensional model processing method, comprising:
The original human face three-dimensional model of face to be modeled is obtained, original human face three-dimensional model includes at least one original top
Point；
Calculate the disparity between original vertices and corresponding standard peak in standard faces threedimensional model；
Geometric transformation is carried out to original vertices to be treated based on disparity, obtaining with the position of standard peak is ginseng
According to representative points；
Original vertices to be treated are replaced with into corresponding representative points in original human face three-dimensional model, are converted
Human face three-dimensional model afterwards.
The above method is reference with a known standard faces threedimensional model, calculates representative points, replaces primitive man
Original vertices to be treated in face three-dimensional model obtain final human face three-dimensional model to be used.Wherein, standard faces are three-dimensional
Model can choose the preferable human face three-dimensional model of quality, and original vertices to be treated, which can choose, causes mould shapes abnormal
Vertex, so that this method can be used for carrying out post-processing optimization to original human face three-dimensional model, treated original face three
Dimension module quality is higher, so as to be applied in various application scenarios, improves user experience.
With reference to first aspect, in the first possible implementation of the first aspect, original vertices and standard people are calculated
After disparity in face three-dimensional model between corresponding standard peak, and based on disparity to be treated original
Before vertex carries out geometric transformation, method further include:
Judge whether disparity is more than preset range of tolerable variance；
If so, original vertices are determined as original vertices to be treated.
If above-mentioned disparity has been more than preset range of tolerable variance, show the original face three-dimensional mould at the original vertices
There are larger differences with the shape of the standard faces threedimensional model at corresponding standard peak for the shape of type, if standard faces three
Dimension module is the preferable human face three-dimensional model of quality, then there are different for shape of the original human face three-dimensional model at the original vertices
Often, it needs to post-process the original vertices to repair this shape anomaly, so that such original vertices are to need to handle
Original vertices.If above-mentioned disparity is less than preset range of tolerable variance, show the original face three at the original vertices
The quality of dimension module is acceptable, without processing.
The possible implementation of with reference to first aspect the first, in second of possible implementation of first aspect
In, for the original vertices in original human face three-dimensional model different zones, the value of range of tolerable variance and original human face three-dimensional model
Curvature value in different zones is positively correlated.
In above-mentioned implementation, different ranges of tolerable variance can be set for original human face three-dimensional model different zones, anticipate
Taste can for different regions characteristic carry out differentiation processing, be conducive to the post-processing to original human face three-dimensional model into
Row more precise control.
Specifically, bright in the biggish region of curvature value of original human face three-dimensional model, such as ear, eyes, nose profile
Aobvious region, disparity between original vertices and standard peak is opposite can be larger, and range of tolerable variance also should be correspondingly at this time
It is arranged larger, avoids excessive original vertices from being post-treated, cause the problem of excessively repairing.Similarly, in original face
The lesser region of the curvature value of threedimensional model, such as cheek, the lower unconspicuous region of profile, original vertices and standard peak it
Between disparity it is opposite can be smaller, range of tolerable variance also correspondingly should be arranged smaller.
With reference to first aspect, in a third possible implementation of the first aspect, disparity includes: original vertices
With displacement difference (X, Y, Z) of the corresponding standard peak in three axis directions, original vertices are with corresponding standard peak in three axis sides
Upward outer corner difference (θ x, θ y, θ z) and original vertices and corresponding standard peak in three axis directions scaling (Sx,
Sy, Sz)；
Geometric transformation is carried out to original vertices to be treated based on disparity, obtaining with the position of standard peak is ginseng
According to representative points, comprising:
Based on two eye distances in original human face three-dimensional model from two eye distances in D1 and standard faces threedimensional model from D2,
Calculate basic scaling Scale1=D1/D2；
Transformation matrix Matrix=M1 (X, Y, Z) * M2 (θ x, θ y, θ z) * M3 (Sx, Sy, Sz) * of computational geometry transformation
Scale1, wherein M1 indicates one using displacement difference as the matrix of element, and M2 expression one is using outer corner difference as the matrix of element, M3
Expression one is using scaling as the matrix of element；
The coordinate Idst=Isrc*Matrix of representative points, wherein Isrc is the coordinate of original vertices to be treated.
With reference to first aspect, in a fourth possible implementation of the first aspect, the original of face to be modeled is obtained
Human face three-dimensional model, comprising:
Obtain multiple depth data frames by depth camera from the faces to be modeled of different station acquisitions；
Determine the depth data frame that can be used for modeling in multiple depth data frames；
Point cloud matching is carried out between corresponding cloud of consecutive frame in the depth data frame that can be used for modeling, and is obtained original
Human face three-dimensional model.
The methods of three-dimensional modeling method, such as structure light in the prior art, need to rely on special hardware device, such as
Structure light device, higher cost, and above-mentioned implementation only needs the depth data frame using depth camera acquisition
Three-dimensional modeling is completed, depth camera is a kind of common device, usually all includes depth camera on current multi-cam mobile phone
Head, cost is relatively low.
The 4th kind of possible implementation with reference to first aspect, in the 5th kind of possible implementation of first aspect
In, while obtaining a depth data frame, also obtain the face to be modeled acquired by common camera from same position
One normal image frame determines the depth data frame that can be used for modeling in multiple depth data frames, comprising:
Face datection is carried out to the corresponding normal image frame of each depth image frame, there are people in determining normal image frame
When face, which is determined as the depth data frame that can be used for modeling.
If face is not detected in normal image frame, indicate in corresponding depth data frame not including face to be modeled
Depth information, which should not be used for face three-dimensional modeling.It is screened first before carrying out face three-dimensional modeling
It can be used for the depth data frame modeled out, be conducive to improve model quality.
The 4th kind of possible implementation with reference to first aspect, in the 6th kind of possible implementation of first aspect
In, point cloud matching is carried out between corresponding cloud of consecutive frame in the depth data frame that can be used for modeling, comprising:
By can be used for model depth data frame be added to point cloud matching queue, the adjacent member of point cloud matching queue it
Between carry out point cloud matching.
With reference to first aspect or first aspect the first to any one possible implementation in the 6th kind,
In 7th kind of possible implementation of one side, standard faces threedimensional model is flat to be constructed based on multiple human face three-dimensional models
Equal model.
Averaging model is counted to a large amount of models as a result, it is possible to represent the normally shape of human face three-dimensional model, is had
There is higher reference value, therefore can be used as mark human face three-dimensional model.
Second aspect, the embodiment of the present invention provide a kind of threedimensional model processing unit, comprising:
Model obtains module, and for obtaining original human face three-dimensional model, original human face three-dimensional model includes at least one original
Beginning vertex；
Disparity computing module, for calculate original vertices and corresponding standard peak in standard faces threedimensional model it
Between disparity；
Representative points computing module is obtained for carrying out geometric transformation to original vertices to be treated based on disparity
The position for being able to standard peak is the representative points of reference；
Object module obtains module, for replacing with original vertices to be treated pair in original human face three-dimensional model
The representative points answered obtain transformed human face three-dimensional model.
The third aspect, the embodiment of the present invention provide a kind of computer readable storage medium, on computer readable storage medium
Computer program instructions are stored with, when computer program instructions are read out by the processor and run, execute first aspect or first party
The step of method that the possible implementation of any one of face provides.
Fourth aspect, the embodiment of the present invention provide a kind of electronic equipment, including memory and processor, deposit in memory
Computer program instructions are contained, when computer program instructions are read out by the processor and run, execute first aspect or first aspect
Any one possible implementation provide method the step of.
To enable above-mentioned purpose of the invention, technical scheme and beneficial effects to be clearer and more comprehensible, special embodiment below, and
Cooperate appended attached drawing, is described in detail below.
Detailed description of the invention
In order to illustrate the technical solution of the embodiments of the present invention more clearly, below will be to needed in the embodiment attached
Figure is briefly described, it should be understood that the following drawings illustrates only certain embodiments of the present invention, therefore is not construed as pair
The restriction of range for those of ordinary skill in the art without creative efforts, can also be according to this
A little attached drawings obtain other relevant attached drawings.
Fig. 1 shows a kind of structural block diagram that can be applied to the electronic equipment in the embodiment of the present invention；
Fig. 2 shows the flow charts of threedimensional model processing method provided in an embodiment of the present invention；
Fig. 3 shows the schematic diagram of the human face three-dimensional model there are quality problems；
Fig. 4 shows the schematic diagram of the second best in quality human face three-dimensional model；
Fig. 5 shows the flow chart of the step S10 of threedimensional model processing method provided in an embodiment of the present invention；
Fig. 6 shows the functional block diagram of threedimensional model processing unit provided in an embodiment of the present invention.
Specific embodiment
Below in conjunction with attached drawing in the embodiment of the present invention, technical solution in the embodiment of the present invention carries out clear, complete
Ground description, it is clear that described embodiments are only a part of the embodiments of the present invention, instead of all the embodiments.Usually exist
The component of the embodiment of the present invention described and illustrated in attached drawing can be arranged and be designed with a variety of different configurations herein.Cause
This, is not intended to limit claimed invention to the detailed description of the embodiment of the present invention provided in the accompanying drawings below
Range, but it is merely representative of selected embodiment of the invention.Based on the embodiment of the present invention, those skilled in the art are not doing
Every other embodiment obtained under the premise of creative work out, shall fall within the protection scope of the present invention.
It should also be noted that similar label and letter indicate similar terms in following attached drawing, therefore, once a certain Xiang Yi
It is defined in a attached drawing, does not then need that it is further defined and explained in subsequent attached drawing.Meanwhile of the invention
In description, term " first ", " second " etc. are only used for distinguishing one entity or operation from another entity or operation,
It is not understood to indicate or imply relative importance, can not be understood as require that or imply and be deposited between these entities or operation
In any actual relationship or order or sequence.Moreover, the terms "include", "comprise" or its any other variant are intended to
Non-exclusive inclusion, so that the process, method, article or equipment including a series of elements is not only wanted including those
Element, but also including other elements that are not explicitly listed, or further include for this process, method, article or equipment
Intrinsic element.In the absence of more restrictions, the element limited by sentence "including a ...", it is not excluded that
There is also other identical elements in process, method, article or equipment including the element.
Fig. 1 shows a kind of structural block diagram that can be applied to the electronic equipment in the embodiment of the present invention.Referring to Fig.1, electronics
Equipment 100 include one or more processors 102, one or more storage device 104, display 106, common camera 108,
Depth camera 110, these components pass through the interconnection of bindiny mechanism's (not shown) of bus system 112 and/or other forms.
Processor 102 can be central processing unit (CPU) or have data-handling capacity and/or instruction execution capability
Other forms processing unit, and can control the other assemblies in electronic equipment 100 to execute desired function.
Storage device 104 can with various forms of computer readable storage mediums, such as volatile memory and/or it is non-easily
The property lost memory.Volatile memory for example may include random access memory (RAM) and/or cache memory
(cache) etc..Nonvolatile memory for example may include read-only memory (ROM), hard disk, flash memory etc..Computer-readable
Can store one or more computer program instructions on storage medium, processor 102 can run computer program instructions, with
Realize the method and/or other desired functions in the embodiment of the present invention.It in a computer-readable storage medium can be with
Store various application programs and various data, such as application program use and/or the various data generated etc..
Display 106 is used to show that user it is expected the various information seen, such as the information such as human face three-dimensional model of foundation,
Show that the content of its 106 display is usually controlled by processor 102.
Common camera 108 is built for acquiring normal image frame, such as RGB image, gray level image etc. as face three-dimensional
For this application scenarios of mould, the content of normal image frame generally should include face to be modeled.
Depth camera 110 be used for sampling depth data frame (i.e. the depth information of scene), as face three-dimensional modeling this
For a application scenarios, the content of depth data frame generally should include the depth information of face to be modeled.
It is appreciated that structure shown in FIG. 1 is only to illustrate, electronic equipment 100 may also include it is more than shown in Fig. 1 or
Less component, or with the configuration different from shown in Fig. 1.Each component shown in Fig. 1 can using hardware, software or its
Combination is realized.In the embodiment of the present invention, electronic equipment 100 may be, but not limited to, desktop computer, laptop, intelligent hand
The common apparatus such as machine, intelligent wearable device are also possible to the professional equipment for face three-dimensional modeling.
Fig. 2 shows the flow charts of threedimensional model processing method provided in an embodiment of the present invention.Below, with the party
Method is applied to be illustrated for the processor 102 of electronic equipment 100, i.e., hereinafter the executing subject of step is electronic equipment
100 processor 102.Referring to Fig. 2, this method comprises:
Step S10: obtaining the original human face three-dimensional model of face to be modeled, and original human face three-dimensional model includes at least one
Original vertices.
Original human face three-dimensional model refers to face three-dimensional mould establishing out by three-dimensional modeling method, without post-processing
Type.The method for establishing original human face three-dimensional model is not construed as limiting, such as can use structure light in the prior art, flight
The methods of time (Time of Flight, abbreviation TOF) can also use point cloud matching method provided in an embodiment of the present invention
(be detailed in and illustrate hereinafter).Due to the limitation of technology, it is possible that all quality as shown in Figure 1 in original human face three-dimensional model
Problem.Referring to Fig. 3, the ratio of entire face is down big up small, and hair mixes with neck, similar to the sample of sphinx
Son, larger with the difference of true face, practical value is lower.
It may include modeling procedure in step S10 in addition it is also necessary to point out, can not also include modeling procedure.
The threedimensional model established in the embodiment of the present invention refers to polygon model, this class model uses a large amount of polygon
The polyhedron approximation that (for example, triangle, quadrangle etc.) is spliced into indicates the profile of object, wherein the vertex of each polygon
The also referred to as vertex of threedimensional model.
Original vertices refer to the vertex of original human face three-dimensional model, and " original " word, which is intended merely to facilitate, distinguishes vertex institute
The model of category, and do not have any Special Significance.The naming method of standard peak, representative points hereinafter is similar.It is right
For threedimensional model, when quoting vertex, is usually quoted by way of vertex index, no longer especially said below
It is bright.
Step S11: the disparity between original vertices and corresponding standard peak in standard faces threedimensional model is calculated.
Standard faces threedimensional model typically refer to quality preferably, the not abnormal human face three-dimensional model of shape, as processing
The reference standard used when original human face three-dimensional model.Standard faces threedimensional model be in advance it is established, for example, can pass through
The mode of manual or automaticization calculates the averaging model for largely constructing good human face three-dimensional model, which can represent
The normally shape of human face three-dimensional model, therefore can be used as standard faces threedimensional model.Certainly, step S11 is actually and unlimited
It is fixed to be selected according to actual needs using which kind of model as standard faces threedimensional model on earth.
Original human face three-dimensional model and standard faces threedimensional model can have identical vertex quantity and identical vertex
Sequentially, so that vertex between the two can correspond.Positional relationship between original vertices and corresponding standard peak
It is defined by disparity, disparity may include translation, rotation, scale caused difference, can be by corresponding public
Formula carries out quantum chemical method.Disparity actually characterize the shape of the original human face three-dimensional model at original vertices with it is corresponding
The similarity degree of the shape of standard faces threedimensional model at standard peak, disparity is bigger, indicates the shape of the two more not
Similar, disparity is smaller, indicates that the shape of the two is more similar.
Step S12: geometric transformation is carried out to original vertices to be treated based on disparity, is obtained with standard peak
Position is the representative points of reference.
All original vertices in original human face three-dimensional model can all be handled, i.e., all by all original vertices
As original vertices to be treated, but for the post-processing task of human face three-dimensional model, usually only need to select one
Dividing causes the vertex of original human face three-dimensional model shape anomaly to be handled, other vertex are without being handled.
For example, a range of tolerable variance can be preset, if some original vertices and corresponding standard peak in step S11
Between calculated disparity be more than the range of tolerable variance, the original vertices are just determined as an original top to be treated
Point.Its principle is:
According to elaboration before, if the disparity between some original vertices and corresponding standard peak has been more than tolerance
Range shows that the shape of the original human face three-dimensional model at the original vertices and the standard faces at corresponding standard peak are three-dimensional
There are larger differences for the shape of model, and standard faces threedimensional model is usually the preferable human face three-dimensional model of quality, therefore can
There is exception with shape of the original human face three-dimensional model of determination at the original vertices, needs to carry out repair process.
If the disparity between some original vertices and corresponding standard peak is less than range of tolerable variance, show that this is original
The shape of the original human face three-dimensional model of apex is not deposited with the shape of the standard faces threedimensional model at corresponding standard peak
In larger difference, there is no need to be further processed.
Certainly, select original vertices to be treated that can also take other modes, such as prior according to statistical law
Which specified good original vertices need to handle.
Disparity based on acquisition can carry out geometric transformation to original vertices, so that after transformation at original vertices
The shape of original human face three-dimensional model is similar with the shape of the standard faces threedimensional model at corresponding standard peak as far as possible, i.e., should
For conversion process using standard peak as reference, transformed original vertices are referred to as representative points.
Step S13: replacing with corresponding representative points for original vertices to be treated in original human face three-dimensional model,
Obtain transformed human face three-dimensional model.
Step S13 is the process that post-processing optimization is carried out to original human face three-dimensional model.For example, if will be several in step S12
What difference is more than the original vertices of range of tolerable variance as original vertices to be treated, after the replacement for executing step S13, by
It is all to be calculated with standard peak for reference, therefore the disparity of replaced vertex and standard peak is usual in representative points
It will fall into range of tolerable variance, so as to eliminate the shape anomaly in original human face three-dimensional model, the primitive man that makes that treated
Face three-dimensional model quality with higher, then be used among subsequent various applications, the experience of user can be improved.It completes
The original human face three-dimensional model of vertex replacement can be as shown in Figure 2.Referring to Fig. 4, the ratio of the face is reasonable, hair, neck etc.
Position separating degree is good, and consistent with true face, practical value is higher.
It should be understood that in the above-mentioned methods, human face three-dimensional model can refer to the head of entire people, it is also possible to finger
The a part in portion.Further, the above method also can be generalized to during the three-dimensional modeling of other objects.
It in one embodiment of the invention, can be with for the original vertices in original human face three-dimensional model different zones
Different ranges of tolerable variance is set, it can carries out differentiation processing for the characteristic in different regions, is conducive to original face
The post-processing of threedimensional model carries out more precise control.
For example, in one implementation, the song in the value of range of tolerable variance and original human face three-dimensional model different zones
Rate value is positively correlated.It is apparent in the biggish region of the curvature value of original human face three-dimensional model, such as ear, eyes, nose profile
Region, through inventor's long-time statistical the study found that disparity between original vertices and standard peak is opposite after modeling can be greatly
Some, range of tolerable variance also correspondingly should be arranged larger at this time, and excessive original vertices is avoided to be post-treated, and cause excessively to repair
Multiple problem, some individualized features for eventually leading to original human face three-dimensional model are erased.Correspondingly, three-dimensional in original face
The lesser region of the curvature value of model, such as cheek, the lower unconspicuous region of profile, between original vertices and standard peak
Disparity is opposite can be smaller, and range of tolerable variance also should be arranged smaller.
In one embodiment of the invention, disparity can be defined as by with three translation, rotation, scaling part groups
At as follows respectively:
Displacement difference (X, Y, Z) of the original vertices with corresponding standard peak in three axis directions；
Outer corner difference (θ x, θ y, θ z) of the original vertices with corresponding standard peak in three axis directions；
Scaling (Sx, Sy, Sz) of the original vertices with corresponding standard peak in three axis directions.
To carry out the geometric transformation in step S13, it is also necessary to calculate original human face three-dimensional model relative to standard faces three
The basic scaling Scale1 of dimension module calculates separately two eye distances in original human face three-dimensional model from D1 and standard faces
Two eye distances in threedimensional model are from D2, then Scale1=D1/D2.
It, can be with the transformation square of computational geometry transformation after obtaining displacement difference, outer corner difference, scaling and basic scaling
Battle array Matrix=M1 (X, Y, Z) * M2 (θ x, θ y, θ z) * M3 (Sx, Sy, Sz) * Scale1, wherein M1 expression one is with displacement difference
For the matrix of element, M2 indicate one using outer corner difference as the matrix of element, M3 indicate one using scaling as the matrix of element,
The concrete form of M1, M2 and M3 can determine according to actual needs.
The coordinate for remembering representative points is Idst, and the coordinate of original vertices to be treated is Isrc, and geometric transformation can lead to
Cross matrix multiplication expression, i.e. Idst=Isrc*Matrix.
In the above-described embodiments, the mathematical description of disparity and geometric transformation is given, certainly in some other reality
It applies in example, however not excluded that define disparity and geometric transformation by other means.
For above-described embodiment, range of tolerable variance involved in step S12 be can be set to for displacement difference, outer corner difference
And the value range of each parameter in scaling, range of tolerable variance can be loaded into before carrying out threedimensional model post-processing.
In one embodiment of the invention, after completing vertex replacement in step s 13, vertex replacement will can be completed
Original human face three-dimensional model is shown on the display 106 of electronic equipment 100, so that user can be with timely learning to original face
It is that threedimensional model is post-processed as a result, to assess threedimensional model processing method provided in an embodiment of the present invention processing effect
Fruit.
In some implementations, can also while handling original human face three-dimensional model just it will be shown in
On display 106, make user that treatment process can be observed in real time.
In one embodiment of the invention, a kind of modeling method of original human face three-dimensional model is also provided.Fig. 5 is shown
The flow chart of the step S10 of threedimensional model processing method provided in an embodiment of the present invention.Referring to Fig. 5, step S10 can be further
Include:
Step S100: multiple depth datas by depth camera 110 from the faces to be modeled of different station acquisitions are obtained
Frame.
Threedimensional model needs to acquire the depth information under multiple angles, these depth informations are spatially associated
It can complete the building of model.The specific acquisition mode of depth data frame is not construed as limiting, for example, can pass through hand-hold electronic equipments
100 are acquired around the mode that face to be modeled is walked about, and electronic equipment 100 can also be placed on rotating mechanism, rotate
Mechanism completes acquisition, etc. around face to be modeled rotation.
Step S101: the depth data frame that can be used for modeling in multiple depth data frames is determined.
If can ensure all include in depth data frame face to be modeled depth information, these depth data frames
It can be used in modeling.But in practice, it is also possible to can collect it is some include face depth data frame, for example, conduct
The people of reference object during the motion, at certain moment, face be likely located at depth camera 110 coverage it
Outside.If not including that the depth data frame of the depth information of face be used to model, model quality decline may cause.Therefore, may be used
To screen to the depth data frame of acquisition, the depth data frame that wherein can be used for modeling is determined.
In some implementations, it can directly detect with the presence or absence of face in depth data frame, only determine wherein
There are when face, just the depth data frame is determined as can be used, otherwise be abandoned.In other implementations, Ke Yitong
The auxiliary for crossing common camera 108 is detected, and in these implementations, electronic equipment 100 includes depth camera simultaneously
110 and common camera 108, the two adjoining position is set.In the same of 110 sampling depth data frame of depth camera
When, common camera 108 also synchronously acquires a normal image frame.Due to depth data frame and corresponding normal image frame
Acquisition position is substantially identical, thus, it is only necessary to it detects in normal image frame with the presence or absence of face, so that it may determine corresponding
It whether there is face in depth data frame.For the Face datection of normal image frame, any one existing face can use
Detection algorithm, here without elaborating.If detecting face in normal image frame, corresponding depth data frame is determined
Be it is available, be otherwise determined as unavailable, which can be abandoned, avoid influencing modeling quality.
Step S102: a cloud is carried out between corresponding cloud of consecutive frame in the depth data frame that can be used for modeling
Match, obtains original human face three-dimensional model.
Each depth data frame corresponds to a piece of cloud, and point cloud matching is exactly to seek spin matrix between two panels point cloud and flat
Matrix is moved, it will be under source point Cloud transform to the identical coordinate system of target point cloud.Point cloud matching can be expressed as equation below:
Pt=R*Ps+T
Wherein Pt is target point cloud, and Ps is source point cloud, and point cloud matching is spin matrix R and the translation matrix asked in the formula
T.The two panels point cloud of consecutive frame can be associated by point cloud matching, corresponding cloud of all consecutive frames is all associated with
Afterwards, actually namely all corresponding depth informations of depth data frame that can be used for modeling are combined together, to complete
The building of original human face three-dimensional model.
Specifically, point cloud matching, which generally includes rough registration, is registrated two steps with essence:
Rough registration is exactly that it is close to find this two panels point cloud in the case where not knowing the relative positional relationship of two panels point cloud completely
As spin matrix R and translation matrix T (not necessarily very accurate, but be probably couple).The evaluation mark of rough registration
It will definitely be to use LCP (Largetst Common Pointset): giving two point sets P and Q, find a transformation T (P), make
The degree of overlapping for obtaining transformed P and Q is maximum.Solve the problems, such as above-mentioned LCP, simplest method is exactly to traverse, multiple to reduce operation
Miscellaneous degree may be incorporated into the strategies such as some search strategies, such as 4PC (4Point Congruent).
Essence registration is exactly (to be obtained by rough registration) in the case where the initial value of known rotation translation, further calculates to obtain more
Add accurate spin matrix R and translation matrix T.By taking ICP (Iterative Closest Point) algorithm as an example, algorithm core
The heart is to minimize an objective function:
In formula,WithIt is a pair of of corresponding points, a total of Np is to corresponding points.This objective function is actually all right
The quadratic sum of Euclidean distance between should putting.Source point cloud is converted with initial spin matrix R and translation matrix T, is obtained
Transformed cloud of one arrived.Then this transformed cloud is compared with target point cloud, if two clouds it
Between two points distance be less than certain threshold value, be considered as the two point be a pair of of corresponding pointsWith.With initial spin moment
Battle array R, translation matrix T and corresponding pointsWithIt is iterated, optimizes R and T, so that the functional value f (R, T) of objective function f is no
Disconnected decline, it is exactly optimal spin matrix R and translation matrix T that ICP algorithm, which finally restrains obtain,.
In a kind of implementation of step S102, as soon as every obtain the depth data frame that can be used for modeling, added
Enter to point cloud matching queue, and its depth data frame that queue was added with last time is subjected to point cloud matching.In some other implementation
In mode, it is also possible to obtain carry out point cloud matching after whole depth data frames again, or deposited using other data structures
Store up depth data frame.
The methods of three-dimensional modeling method, such as structure light in the prior art, need to rely on special hardware device, such as
Structure light device, higher cost, and the depth data frame for only needing to acquire using depth camera 110 in above-described embodiment is i.e.
Achievable three-dimensional modeling, depth camera 110 are a kind of common devices, usually all include depth on current multi-cam mobile phone
Camera 110 and common camera 108, cost is relatively low for implementation, easy to spread universal.
The embodiment of the present invention also provides a kind of threedimensional model processing unit 200.Referring to Fig. 6, which includes:
Model obtains module 210, and for obtaining original human face three-dimensional model, original human face three-dimensional model includes at least one
Original vertices；
Disparity computing module 220, for calculating original vertices and corresponding standard top in standard faces threedimensional model
Disparity between point；
Representative points computing module 230, for carrying out geometric transformation to original vertices to be treated based on disparity,
It obtains using the position of standard peak as the representative points of reference；
Object module obtains module 240, for replacing original vertices to be treated in original human face three-dimensional model
For corresponding representative points, transformed human face three-dimensional model is obtained.
The technical effect of threedimensional model processing unit 200 provided in an embodiment of the present invention, realization principle and generation is with before
It is identical to state embodiment of the method, to briefly describe, Installation practice part does not refer to place, can refer to preceding method and applies phase in example
Answer content.
The embodiment of the present invention also provides a kind of computer readable storage medium, and meter is stored on computer readable storage medium
Calculation machine program instruction when computer program instructions are read out by the processor and run, executes three-dimensional mould provided in an embodiment of the present invention
The step of type processing method.The computer readable storage medium can be implemented as, but be not limited to storage device 104 shown in fig. 1.
The embodiment of the present invention also provides a kind of electronic equipment, including memory and processor, is stored with meter in memory
Calculation machine program instruction when computer program instructions are read out by the processor and run, executes three-dimensional mould provided in an embodiment of the present invention
The step of type processing method.The electronic equipment can be implemented as, but be not limited to electronic equipment 100 shown in fig. 1.
It should be noted that all the embodiments in this specification are described in a progressive manner, each embodiment weight
Point explanation is the difference from other embodiments, and the same or similar parts between the embodiments can be referred to each other.
For device class embodiment, since it is basically similar to the method embodiment, so being described relatively simple, related place ginseng
See the part explanation of embodiment of the method.
In several embodiments provided herein, it should be understood that disclosed device and method can also pass through it
His mode is realized.The apparatus embodiments described above are merely exemplary, for example, the flow chart and block diagram in attached drawing are aobvious
The device of multiple embodiments according to the present invention, architectural framework in the cards, the function of method and computer program product are shown
It can and operate.In this regard, each box in flowchart or block diagram can represent one of a module, section or code
Point, a part of the module, section or code includes one or more for implementing the specified logical function executable
Instruction.It should also be noted that function marked in the box can also be attached to be different from some implementations as replacement
The sequence marked in figure occurs.For example, two continuous boxes can actually be basically executed in parallel, they sometimes may be used
To execute in the opposite order, this depends on the function involved.It is also noted that each of block diagram and or flow chart
The combination of box in box and block diagram and or flow chart can be based on the defined function of execution or the dedicated of movement
The system of hardware is realized, or can be realized using a combination of dedicated hardware and computer instructions.
In addition, each functional module in each embodiment of the present invention can integrate one independent portion of formation together
Point, it is also possible to modules individualism, an independent part can also be integrated to form with two or more modules.
It, can be with if the function is realized and when sold or used as an independent product in the form of software function module
It is stored in computer-readable storage medium.Based on this understanding, technical solution of the present invention is substantially in other words to existing
Having the part for the part or the technical solution that technology contributes can be embodied in the form of software products, the computer
Software product is stored in a storage medium, including some instructions are used so that computer equipment executes each embodiment institute of the present invention
State all or part of the steps of method.Computer equipment above-mentioned includes: personal computer, server, mobile device, intelligently wears
The various equipment with execution program code ability such as equipment, the network equipment, virtual unit are worn, storage medium above-mentioned includes: U
Disk, mobile hard disk, read-only memory, random access memory, magnetic disk, tape or CD etc. are various to can store program code
Medium.
The above description is merely a specific embodiment, but scope of protection of the present invention is not limited thereto, any
Those familiar with the art in the technical scope disclosed by the present invention, can easily think of the change or the replacement, and should all contain
Lid is within protection scope of the present invention.Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (11)
1. a kind of threedimensional model processing method characterized by comprising
The original human face three-dimensional model of face to be modeled is obtained, the original human face three-dimensional model includes at least one original top
Point；
Calculate the disparity between the original vertices and corresponding standard peak in standard faces threedimensional model；
Geometric transformation is carried out to original vertices to be treated based on the disparity, is obtained with the position of the standard peak
For the representative points of reference；
The original vertices to be treated are replaced with into the corresponding representative points in the original human face three-dimensional model,
Obtain transformed human face three-dimensional model.
2. threedimensional model processing method according to claim 1, which is characterized in that described to calculate the original vertices and mark
After disparity in quasi- human face three-dimensional model between corresponding standard peak and it is described based on the disparity to need
Before original vertices to be processed carry out geometric transformation, the method also includes:
Judge whether the disparity is more than preset range of tolerable variance；
If so, the original vertices are determined as the original vertices to be treated.
3. threedimensional model processing method according to claim 2, which is characterized in that be directed to the original human face three-dimensional model
In the original vertices in different zones, the value of the range of tolerable variance and the original human face three-dimensional model different zones
Curvature value is positively correlated.
4. threedimensional model processing method according to claim 1, which is characterized in that the disparity includes: the original
Displacement difference (X, Y, Z) of the beginning vertex with the corresponding standard peak in three axis directions, the original vertices and corresponding institute
State outer corner difference (θ x, θ y, θ z) of the standard peak in three axis directions and the original vertices and the corresponding standard peak
Scaling (Sx, Sy, Sz) in three axis directions；
It is described that geometric transformation is carried out to original vertices to be treated based on the disparity, it obtains with the standard peak
Position is the representative points of reference, comprising:
Based on two eye distances in the original human face three-dimensional model from two eye distances in D1 and the standard faces threedimensional model
From D2, basic scaling Scale1=D1/D2 is calculated；
Transformation matrix Matrix=M1 (X, Y, Z) * M2 (θ x, θ y, θ z) * M3 (Sx, Sy, Sz) * Scale1 of computational geometry transformation,
Wherein, M1 indicates one using the displacement difference as the matrix of element, and M2 expression one is using the outer corner difference as the matrix of element, M3
Expression one is using the scaling as the matrix of element；
The coordinate Idst=Isrc*Matrix of the representative points, wherein Isrc is the seat of the original vertices to be treated
Mark.
5. threedimensional model processing method according to claim 1, which is characterized in that described to obtain the original of face to be modeled
Human face three-dimensional model, comprising:
It obtains as depth camera from multiple depth data frames of face to be modeled described in different station acquisitions；
Determine the depth data frame that can be used for modeling in the multiple depth data frame；
Point cloud matching is carried out between corresponding cloud of consecutive frame in the depth data frame that can be used for modeling, described in acquisition
Original human face three-dimensional model.
6. threedimensional model processing method according to claim 5, which is characterized in that obtaining the same of a depth data frame
When, also obtain a normal image frame as common camera from face to be modeled described in same position acquisition, the determination
It can be used for the depth data frame modeled in the multiple depth data frame, comprising:
The normal image frame corresponding to each depth image frame carries out Face datection, deposits in determining the normal image frame
In face, which is determined as the depth data frame that can be used for modeling.
7. threedimensional model processing method according to claim 5, which is characterized in that described in the depth that can be used for modeling
Point cloud matching is carried out between corresponding cloud of consecutive frame in degree data frame, comprising:
By it is described can be used for model depth data frame be added to point cloud matching queue, the point cloud matching queue it is adjacent at
Point cloud matching is carried out between member.
8. threedimensional model processing method described in any one of -7 according to claim 1, which is characterized in that the standard faces
Threedimensional model is the averaging model constructed based on multiple human face three-dimensional models.
9. a kind of threedimensional model processing unit characterized by comprising
Model obtains module, and for obtaining original human face three-dimensional model, the original human face three-dimensional model includes at least one original
Beginning vertex；
Disparity computing module, for calculate the original vertices and corresponding standard peak in standard faces threedimensional model it
Between disparity；
Representative points computing module is obtained for carrying out geometric transformation to original vertices to be treated based on the disparity
The position for being able to the standard peak is the representative points of reference；
Object module obtains module, for replacing the original vertices to be treated in the original human face three-dimensional model
For the corresponding representative points, transformed human face three-dimensional model is obtained.
10. a kind of computer readable storage medium, which is characterized in that be stored with computer on the computer readable storage medium
Program instruction, when the computer program instructions are read out by the processor and run, perform claim is required described in any one of 1-8
Method the step of.
11. a kind of electronic equipment, which is characterized in that including memory and processor, be stored with computer in the memory
Program instruction, when the computer program instructions are read and run by the processor, perform claim requires any one of 1-8
The step of described method.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811236759.4A CN109300188A (en) | 2018-10-23 | 2018-10-23 | Threedimensional model processing method and processing device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811236759.4A CN109300188A (en) | 2018-10-23 | 2018-10-23 | Threedimensional model processing method and processing device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109300188A true CN109300188A (en) | 2019-02-01 |
Family
ID=65157640
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811236759.4A Pending CN109300188A (en) | 2018-10-23 | 2018-10-23 | Threedimensional model processing method and processing device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109300188A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110059660A (en) * | 2019-04-26 | 2019-07-26 | 北京迈格威科技有限公司 | Mobile terminal platform 3D face registration method and device |
| CN111127668A (en) * | 2019-12-26 | 2020-05-08 | 网易（杭州）网络有限公司 | Role model generation method and device, electronic equipment and storage medium |
| CN117351170A (en) * | 2023-10-09 | 2024-01-05 | 北京达美盛软件股份有限公司 | Method and system for realizing regional three-dimensional model replacement |
| CN117612676A (en) * | 2023-11-08 | 2024-02-27 | 中国人民解放军总医院第四医学中心 | Method and device for realizing batch extraction of human anatomy characteristic parameters |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104050712A (en) * | 2013-03-15 | 2014-09-17 | 索尼公司 | Method and apparatus for establishing three-dimensional model |
| CN105989604A (en) * | 2016-02-18 | 2016-10-05 | 合肥工业大学 | Target object three-dimensional color point cloud generation method based on KINECT |
| CN107292921A (en) * | 2017-06-19 | 2017-10-24 | 电子科技大学 | A kind of quick three-dimensional reconstructing method based on kinect cameras |
| CN108447017A (en) * | 2018-05-31 | 2018-08-24 | Oppo广东移动通信有限公司 | Face virtual face-lifting method and device |
| CN108550185A (en) * | 2018-05-31 | 2018-09-18 | Oppo广东移动通信有限公司 | Beautifying faces treating method and apparatus |
-
2018
- 2018-10-23 CN CN201811236759.4A patent/CN109300188A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104050712A (en) * | 2013-03-15 | 2014-09-17 | 索尼公司 | Method and apparatus for establishing three-dimensional model |
| CN105989604A (en) * | 2016-02-18 | 2016-10-05 | 合肥工业大学 | Target object three-dimensional color point cloud generation method based on KINECT |
| CN107292921A (en) * | 2017-06-19 | 2017-10-24 | 电子科技大学 | A kind of quick three-dimensional reconstructing method based on kinect cameras |
| CN108447017A (en) * | 2018-05-31 | 2018-08-24 | Oppo广东移动通信有限公司 | Face virtual face-lifting method and device |
| CN108550185A (en) * | 2018-05-31 | 2018-09-18 | Oppo广东移动通信有限公司 | Beautifying faces treating method and apparatus |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110059660A (en) * | 2019-04-26 | 2019-07-26 | 北京迈格威科技有限公司 | Mobile terminal platform 3D face registration method and device |
| CN111127668A (en) * | 2019-12-26 | 2020-05-08 | 网易（杭州）网络有限公司 | Role model generation method and device, electronic equipment and storage medium |
| CN111127668B (en) * | 2019-12-26 | 2023-08-22 | 网易（杭州）网络有限公司 | Character model generation method and device, electronic equipment and storage medium |
| CN117351170A (en) * | 2023-10-09 | 2024-01-05 | 北京达美盛软件股份有限公司 | Method and system for realizing regional three-dimensional model replacement |
| CN117612676A (en) * | 2023-11-08 | 2024-02-27 | 中国人民解放军总医院第四医学中心 | Method and device for realizing batch extraction of human anatomy characteristic parameters |
| CN117612676B (en) * | 2023-11-08 | 2024-06-07 | 中国人民解放军总医院第四医学中心 | Method and device for realizing batch extraction of human anatomy characteristic parameters |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108416840B (en) | Three-dimensional scene dense reconstruction method based on monocular camera | |
| CN109300188A (en) | Threedimensional model processing method and processing device | |
| US20210165245A1 (en) | Method, device, and computer program for virtually adjusting a spectacle frame | |
| US6434278B1 (en) | Generating three-dimensional models of objects defined by two-dimensional image data | |
| CN107945267A (en) | A kind of method and apparatus for human face three-dimensional model grain table | |
| CN109859296A (en) | Training method, server and the storage medium of SMPL parametric prediction model | |
| CN111784821B (en) | Three-dimensional model generation method and device, computer equipment and storage medium | |
| CN107358648A (en) | Real-time full-automatic high quality three-dimensional facial reconstruction method based on individual facial image | |
| CN110909611A (en) | Method and device for detecting attention area, readable storage medium and terminal equipment | |
| JP2001283229A (en) | Method for calculating position and direction of object in three-dimensional space | |
| CN107967693A (en) | Video Key point processing method, device, computing device and computer-readable storage medium | |
| CN114842121B (en) | Method, device, equipment and medium for generating mapping model training and mapping | |
| CN106033621A (en) | Three-dimensional modeling method and device | |
| CN110070610B (en) | Feature point matching method, and feature point matching method and device in three-dimensional reconstruction process | |
| Lawonn et al. | Visualization and extraction of carvings for heritage conservation | |
| CN112070782A (en) | Method and device for identifying scene contour, computer readable medium and electronic equipment | |
| CN112329726B (en) | Face recognition method and device | |
| CN109766896B (en) | Similarity measurement method, device, equipment and storage medium | |
| Canessa et al. | A dataset of stereoscopic images and ground-truth disparity mimicking human fixations in peripersonal space | |
| CN108573192B (en) | Glasses try-on method and device matched with human face | |
| CN110647790A (en) | Method and device for determining gazing information | |
| CN109191447B (en) | Three-dimensional grid quality evaluation method based on geometric curvature analysis | |
| CN109544619A (en) | A kind of binocular vision solid matching method cut based on figure and system | |
| CN115375836A (en) | Point cloud fusion three-dimensional reconstruction method and system based on multivariate confidence filtering | |
| Maghoumi et al. | Gemsketch: Interactive image-guided geometry extraction from point clouds |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190201 |