CN108921908A - Acquisition method, device and the electronic equipment of surface optical field - Google Patents
Acquisition method, device and the electronic equipment of surface optical field Download PDFInfo
- Publication number
- CN108921908A CN108921908A CN201810720340.XA CN201810720340A CN108921908A CN 108921908 A CN108921908 A CN 108921908A CN 201810720340 A CN201810720340 A CN 201810720340A CN 108921908 A CN108921908 A CN 108921908A
- Authority
- CN
- China
- Prior art keywords
- image
- patch grids
- grids
- patch
- target image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/90—Determination of colour characteristics
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
Abstract
The embodiment of the invention discloses a kind of acquisition method of surface optical field, device and electronic equipment, the method includes：Obtain multiple images of each patch grids in different perspectives on object model surface；For each patch grids, an image is obtained from multiple images of patch grids as target image, remaining each image in patch grids in addition to the target image is aligned with target image, generates the image collection of patch grids；Each image in each image collection is sampled, so that the pixel of each image in each image collection is identical, and visual angle interval is equal；Image collection after the corresponding sampling of all patch grids is decomposed and compressed, to obtain the surface optical field data of object model.The application is aligned sampled images before surface optical field is compressed, and then eliminates the influence of inaccurate geometry, to reduce surface optical field sampling and handle the dependence to precise geometrical model.
Description
Technical field
The present embodiments relate to technical field of image processing more particularly to a kind of acquisition methods of surface optical field, device
And electronic equipment.
Background technique
It is (both so-called that light field Rendering is generally used for capturing, record, compress and storing all light around target object
Light field), and in rendering, angle restores all light informations according to the observation, so that precise restoration target object is in real world
In it is apparent.Because light field Rendering avoids the material estimation to target object and is more stranded to ambient lighting estimation etc. is multiple
Difficult problem, light field can effectively rebuild the apparent of real world objects, and realize that photorealistic renders, can be in many
It plays a significant role in VR/AR application.
Surface optical field is a branch of light field, has been pushed further by the geological information using target object traditional
Light field Rendering, can in bigger visible angle (360 °) undistorted record light field.Therefore, surface optical field be used to restore one
A bit with the object of more complicated appearance and geometrical model.However, surface optical field is dependent on an accurate geometrical model to light field
It is sampled and is handled.And being widely used which has limited it because the geometric jacquard patterning unit surface of many target objects be all not easy using
The reconstructing apparatus of conventional precision carries out Exact recovery.
Summary of the invention
The embodiment of the present invention provides the acquisition method, device and electronic equipment of a kind of surface optical field.
In a first aspect, the embodiment of the present invention provides a kind of acquisition method of surface optical field, including：
Multiple images of each patch grids in different perspectives on object model surface are obtained, wherein the object model
For the model for formed after grid dividing by the surface of object；
For each patch grids, an image is obtained from multiple images of the patch grids as target figure
Remaining each image in the patch grids in addition to the target image is aligned, described in generation by picture with the target image
The image collection of patch grids；
Each image in each described image set is sampled, so that each image in each described image set
Pixel is identical, and visual angle interval is equal；
Described image set after the corresponding sampling of all patch grids is decomposed and compressed, determines the object mould
The surface optical field data of type.
It is described for each patch grids in a kind of possible implementation of first aspect, from the dough sheet
An image is obtained in multiple images of grid as target image, including：
For each patch grids, by the maximum image of diffusion color value in the corresponding each image of the patch grids
Target image as the patch grids.
It is described for each patch grids in the alternatively possible implementation of first aspect, from the face
An image is obtained in multiple images of piece grid as target image, including：
Determine the sum of the energy of the corresponding all images of all patch grids；
Using each patch grids corresponding image when the sum of described energy minimum as the target image of each patch grids.
In the alternatively possible implementation of first aspect, the corresponding each image of each patch grids of the determination
The sum of energy, including：
According to formulaDetermine each patch grids
The energy summation E (P) of every image；
Wherein, describedIt is described for the color value of i-th image of patch grids fIt is describedIt is corresponding bright
Angle value, it is describedIt is describedCorresponding sample quality, the f ' are the adjacent patch grids of the patch grids f, institute
It statesIt is described for the color value of the jth image of the patch grids f 'For the patch grids f and the face
Piece grid f ' shares the color difference on side.
It is described that the target image will be removed in the patch grids in the alternatively possible implementation of first aspect
Except remaining each image be aligned with the target image, including：
Determine remaining each image of the patch grids and the similar energies value of the target image；
When the similar energies value maximum, remaining each image and the target image pair of the patch grids are determined
Together.
In the alternatively possible implementation of first aspect, remaining each image of the determination patch grids with
The similar energies value of the target image, including：
According to formulaDetermine remaining each image of the patch grids with it is described
The similar energies value E of target imagef(Df,t)；
Wherein, the DfIt is described for the target image of the patch grids fFor the face of i-th image of patch grids f
Color value, the tiFor the translational movement of i-th image of the patch grids f.
In the alternatively possible implementation of first aspect, remaining each image of the determination patch grids with
The similar energies value of the target image, including：
According to formulaDetermine remaining of the patch grids
The similar energies value E of each image and the target imagef(Df,t)；
Wherein, the DfIt is described for the target image of the patch grids fFor the face of i-th image of patch grids f
Color value, the tiFor the translational movement of i-th image of the patch grids f, the t0For preset value.
In the alternatively possible implementation of first aspect, which is characterized in that the patch grids is gore
Piece grid.
Second aspect, the embodiment of the present invention provide a kind of acquisition device of surface optical field, including：
First obtains module, for obtaining multiple figures of each patch grids in different perspectives on object model surface
Picture, wherein the object model is the model for formed after grid dividing by the surface of object；
Second obtains module, for being obtained from multiple images of the patch grids for each patch grids
One image is as target image；
Alignment module, for by the patch grids in addition to the target image remaining each image and the target
Image alignment generates the image collection of the patch grids；
Sampling module, for being sampled to each image in each described image set, so that each described image collection
The pixel of each image in conjunction is identical, and visual angle interval is equal；
Compression module is decomposed, for the described image set after the corresponding sampling of all patch grids to be decomposed and pressed
Contracting, determines the surface optical field data of the object model.
In a kind of possible implementation of second aspect, described second obtains module, for for each face
Piece grid, using the maximum image of diffusion color value in the corresponding each image of the patch grids as the target of the patch grids
Image.
In the alternatively possible implementation of second aspect, the second acquisition module includes：
First computing unit, for determining the sum of the energy of the corresponding all images of all patch grids；
Second determination unit, for using each patch grids corresponding image when the sum of described energy minimum as each face
The target image of piece grid.
In the alternatively possible implementation of second aspect, first computing unit is specifically used for according to formulaDetermine that the energy of every image of each patch grids is total
With E (P)；
Wherein, describedIt is described for the color value of i-th image of patch grids fIt is describedIt is corresponding bright
Angle value, it is describedIt is describedCorresponding sample quality, the f ' are the adjacent patch grids of the patch grids f, institute
It statesIt is described for the color value of the jth image of the patch grids f 'For the patch grids f and the face
Piece grid f ' shares the color difference on side.
In the alternatively possible implementation of second aspect, the alignment module includes：
Second computing unit, the similar energies of remaining each image and the target image for determining the patch grids
Value；
Second determination unit, for determining remaining each image of the patch grids when the similar energies value maximum
It is aligned with the target image.
In the alternatively possible implementation of second aspect, second computing unit is specifically used for according to formulaDetermine that remaining each image of the patch grids is similar to the target image
Energy value Ef(Df,t)；
Wherein, the DfIt is described for the target image of the patch grids fFor the face of i-th image of patch grids f
Color value, the tiFor the translational movement of i-th image of the patch grids f.
In the alternatively possible implementation of second aspect, second computing unit, also particularly useful for according to public affairs
FormulaDetermine remaining each image and the mesh of the patch grids
The similar energies value E of logo imagef(Df,t)；
Wherein, the DfIt is described for the target image of the patch grids fFor the face of i-th image of patch grids f
Color value, the tiFor the translational movement of i-th image of the patch grids f, the t0For preset value.
In the alternatively possible implementation of second aspect, the patch grids is triangle surface grid.
The third aspect, the embodiment of the present invention provide a kind of electronic equipment, including：
Memory, for storing computer program；
Processor, for executing the computer program, to realize the acquisition method of surface optical field described in first aspect.
Fourth aspect, the embodiment of the present invention provide a kind of computer storage medium, store computer in the storage medium
Program, the computer program are used to execute the acquisition method of surface optical field described in first aspect.
Acquisition method, device and the electronic equipment of surface optical field provided in an embodiment of the present invention, by obtaining object model
Multiple images of each patch grids in different perspectives on surface, wherein the object model is that the surface of object is carried out net
The model that lattice are formed after dividing；For each patch grids, a figure is obtained from multiple images of the patch grids
As being used as target image, by remaining each image and the target image pair in the patch grids in addition to the target image
Together, the image collection of the patch grids is generated；Each image in each described image set is sampled, so that each institute
The pixel for stating each image in image collection is identical, and visual angle interval is equal；To the institute after the corresponding sampling of all patch grids
It states image collection to be decomposed and compressed, determines the surface optical field data of the object model.That is the present embodiment, in surface optical field
Before carrying out compression process, sampled images are aligned, and then eliminate the influence of inaccurate geometry, to reduce surface
Light field sampling and processing are to the dependence of precise geometrical model.
Detailed description of the invention
In order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology, to embodiment or will show below
There is attached drawing needed in technical description to be briefly described, it should be apparent that, the accompanying drawings in the following description is this hair
Bright some embodiments for those of ordinary skill in the art without any creative labor, can be with
It obtains other drawings based on these drawings.
Fig. 1 is the flow diagram of the acquisition method for the surface optical field that the embodiment of the present invention one provides；
Fig. 2 is the part of the surface light field data schematic diagram that the embodiment of the present invention one is related to；
Fig. 3 is the schematic diagram of selection target image from unjustified each image that the embodiment of the present invention one is related to；
Fig. 4 is each image schematic diagram after the alignment that the embodiment of the present invention one is related to；
Fig. 5 is the surface optical field schematic diagram data optimized to surface optical field data shown in Fig. 2；
Fig. 6 is the flow example figure of the acquisition method of surface optical field provided by Embodiment 2 of the present invention；
Fig. 7 is the flow example figure of the acquisition method for the surface optical field that the embodiment of the present invention three provides；
Fig. 8 is the structural schematic diagram of the acquisition device for the surface optical field that the embodiment of the present invention one provides；
Fig. 9 is the structural schematic diagram of the acquisition device of surface optical field provided by Embodiment 2 of the present invention；
Figure 10 is the structural schematic diagram of the acquisition device for the surface optical field that the embodiment of the present invention three provides；
Figure 11 is the structural schematic diagram of electronic equipment provided in an embodiment of the present invention.
Specific embodiment
In order to make the object, technical scheme and advantages of the embodiment of the invention clearer, below in conjunction with the embodiment of the present invention
In attached drawing, technical scheme in the embodiment of the invention is clearly and completely described, it is clear that described embodiment is
A part of the embodiment of the present invention, instead of all the embodiments.Based on the embodiments of the present invention, those of ordinary skill in the art
Every other embodiment obtained without creative efforts, shall fall within the protection scope of the present invention.
Surface optical field is sampled and is handled to light field dependent on an accurate geometrical model.And which has limited the wide of it
General use, because the geometric jacquard patterning unit surface of many target objects is all not easy to carry out Exact recovery using the reconstructing apparatus of conventional precision
To solve this technical problem, the application proposes a kind of acquisition method of surface optical field, is not direct optimization geometry knot
Structure, to realize the modeling and rendering of steady and high-fidelity, but by each patch grids on object model surface in difference
Multiple images on visual angle are aligned, and to optimize these sampled points, and then eliminate the influence of inaccurate geometry.Then,
Image after alignment is compressed, the surface optical field data of object model are generated.
I.e. embodiment decompose with before compression process in surface optical field, is aligned to sampled images, and then eliminate
The influence of inaccurate geometry, to reduce surface optical field sampling and handle the dependence to precise geometrical model.
Technical solution of the present invention is described in detail with specifically embodiment below.These specific implementations below
Example can be combined with each other, and the same or similar concept or process may be repeated no more in some embodiments.
Fig. 1 is the flow diagram of the acquisition method for the surface optical field that the embodiment of the present invention one provides.Should as shown in Figure 1,
The method of the present embodiment may include：
S101, multiple images of each patch grids in different perspectives on object model surface mesh are obtained, wherein institute
Stating object model is the model for formed after patch grids division by the surface of object.
The executing subject of the present embodiment can be the acquisition device that the surface optical field of function is determined with surface optical field data,
The acquisition device of the surface optical field of the present embodiment can be a part of electronic equipment, for example, processor of electronic equipment.It can
The acquisition of the surface optical field of the present embodiment of choosing can also be individual electronic equipment.
The electronic equipment of the present embodiment can be smart phone, desktop computer, laptop, Intelligent bracelet, AR equipment,
The electronic equipments such as VA equipment.
The present embodiment is illustrated so that executing subject is electronic equipment as an example.
Surface optical field is a kind of Rendering based on image, is used for visualization result.It is established on polygon curved surface new
Representation, be used for actual data sampling.Space sample is placed on vertex, and the interpolation on triangle.And from screen
Polygon subdivision direction sample in curtain space.Since the sampled data in the expression is stored as pattern matrix, block is utilized
Coding techniques is compressed.
Before the acquisition for carrying out surface optical field, it is necessary first to carry out grid dividing to the surface of object, generate object mould
Block.
Optionally, the grid that any shape can be used in the present embodiment divides object, for example, using quadrangle,
The shape changeables such as pentagon, hexagon patch grids divides the surface of object.
In a kind of example, in order to reduce the difficulty of division, triangle surface grid is can be used to object in the present embodiment
Surface carry out grid dividing.
In this way, forming the surface of object by patch grids.
The size of each patch grids can be the same or different, and same object can be used the grid of same shape
It is divided, grid dividing of different shapes also can be used, the present embodiment is without limitation, true with specific reference to actual demand
It is fixed.
It for each patch grids on object model, takes pictures in different perspectives to patch grids, obtains dough sheet
Multiple images of grid in different perspectives.
For example, taking pictures from 3 different visual angles to patch grids A, 3 of patch grids A on 3 visual angles are obtained
Open image.
In practical applications, it is assumed that capture object with surface mesh M, grid dividing is carried out to object, forms object mould
Type, wherein the surface of object model is by NfA patch grids composition, i.e. patch grids set F={ fi... fNfComposition.
Then, object model is placed on to the center of calibration plate, in different perspectives, shoots multiple images I={ I1...
.IN}.Meanwhile extracting corresponding camera external parameter.
Optionally, due to only having parts of images to target face as it can be seen that extracting from the image of each patch grids f ∈ F
Part dough sheet, first progress visibility inspection are found one to the visible image subset of face, are incited somebody to action using corresponding camera parameter
Face back projection is into image space, to generate one group of image Pf={ Pf 1,…Pf n}.Here, n is equal to the visual picture of face
Number.
S102, for each patch grids, an image conduct is obtained from multiple images of the patch grids
Remaining each image in the patch grids in addition to the target image is aligned by target image with the target image, raw
At the image collection of the patch grids.
As shown in Fig. 2, traditional direct compression may cause artifact when geometry is obviously inaccurate.
In order to solve the technical problem, the present embodiment optimizes patch grids before light field decomposition, inaccurate several to eliminate
The negative effect of what shape.
Specifically, according to above-mentioned steps, it is corresponding to each patch grids after multiple images for obtaining each patch grids
Multiple images be aligned, such as by 3 image alignments of patch grids A.
In alignment procedure, illustrate by taking patch grids A as an example, is selected from multiple corresponding images of patch grids A first
One target image as patch grids A.Such as shown in Fig. 3, dark-colored image is the target image of patch grids A.
Then, the remaining image in patch grids A in addition to target image is aligned with the target image, example
Such as, 2D is introduced in space translate ti, calculate the translation t in patch grids A between remaining each image and target imagei, will
Remaining image is aligned with the target image.Such as shown in Fig. 4, by the target figure of the remaining image of patch grids A and dark-colored image
As alignment.
In this way, the set that each image after patch grids A alignment is formed is as patch grids A image collection.
Referring to above-mentioned steps, the image collection of each patch grids can be obtained.
In one possible implementation, above-mentioned for each patch grids, from multiple of the patch grids
An image is obtained in image as target image, can be：
For each patch grids, by the maximum image of diffusion color value in the corresponding each image of the patch grids
Target image as the patch grids.
For example, determining the diffusion color value of every image for patch grids A, the maximum image of diffusion color value is made
For the target image of patch grids A.
In this way, the corresponding target image of each patch grids can be obtained.
S103, each image in each described image set is sampled, so that each in each described image set
The pixel of image is identical, and visual angle interval is equal.
Since the pixel (i.e. size) of each image of above-mentioned alignment may be different, and when shooting image, the angle of camera
Nor it is equally spaced, therefore, before being decomposed, need to sample each image after alignment, specifically, so that together
The pixel of each image in one image collection is identical, and visual angle interval is equal.
In this way, the pixel of each image in the same image collection is identical, and visual angle interval is equal, is carrying out subsequent point
When solution and compression, the accuracy of decomposition and compression can be improved, so that the surface optical field data ultimately produced are more accurate.
S104, the described image set after the corresponding sampling of all patch grids is decomposed and is compressed, described in determination
The surface optical field data of object model.
Specifically, being sampled according to above-mentioned steps to the image after the alignment of each patch grids, so that the pixel of each image
Identical, visual angle interval is equal.Then, the image collection after the corresponding sampling of all patch grids is decomposed, it is then right
Image after decomposition is compressed, and generates the high-precision surface optical field data of object model, and carry out to the surface optical field data
It saves.
Fig. 5 is the schematic diagram compressed after each image corresponding to Fig. 2 is aligned, as shown in figure 5, pressing again after alignment
Contracting can effectively reduce ghost image, improve the precision of surface optical field data.
In VR (Virtual Reality, virtual reality) or AR (Augmented Reality, augmented reality) technology,
The surface optical field data that above-mentioned determination can be used render object, object to be presented in virtual environment in true environment
In light-field effects, to improve user experience.
Wherein, the present embodiment image collection is decomposed and the mode compressed with no restrictions, specifically used existing
One mode.
In a kind of example, surface optical field can be expressed as four-dimensional function L (u, v, s, t), wherein (u, v) is on surface
Position, (s, t) are view direction.
Light field function can be further broken into a small amount of sum of products of low-dimensional function：
L(u,v,s,t)≈∑S(u,v)V(s,t) (1)
Wherein, S (u, v) is toroidal function, V (s, t) view functions.
This decomposition attempts to separate the variation of the variation of surface texture and illumination.These functions can be by using
PCA (Principal Component Analysis, principal component analysis) or nonlinear optimization construct.Functional parameter can be deposited
Storage is in texture mapping and real-time rendering.
In order to be easily achieved curved surface light field under rendering pipeline, L (u, v, s, t) is made to cross over small surface element, and
It independently is each part and establishes approximation.Specifically, it samples and constructs in an annular dough sheet net region by vertex x
One group of vertex light field Lx(u,v,s,t)。
In the implementation, vertex light field function Lx(u, v, s, t) can be expressed as matrix Lx[u,v,s,t]∈Rm×n, in curved surface
Discretization on piece and perspective view.Matrix column n indicates camera view, and row m indicates surface location.Storage matrix Lx's
Full set be it is unpractical, need to decompose light field data and compressed.In addition, according to the reason of dichromatic reflection theory
By, it is also necessary to from matrix LxMiddle separation diffuses component Dx[s, t] and LxResidual components Gx[s,t,u,v].To remainder
Carry out following conventional compact：
Lx[u, v, s, t]=Dx[u,v]+Gx[u, v, s, t]=Dx[u,v]+∑Sx[u,v]Vx[s,t] (2)
Wherein, Sx[u, v] is the surface mapping matrix of vertex x, Vx[s, t] is the View Mapping matrix of vertex x, Ke Yicong
Discretization in surface and view functions in formula (1).(u, v) is synthesized by its an annular triangle neighbours by vertex
Space coordinate in dough sheet.[s, t] is the view coordinate in hemisphere harmonic wave.
In a kind of example, the G of SVD (Singular Value Decomposition, singular value decomposition) is usedx=
Sx.VxBy the residual color G of resamplingx(residual components of i.e. above-mentioned Lx) are decomposed into k exterior views and view.Wherein Sx
It is m × k matrix of left singular vector, will be multiplied by the singular value diagonal matrix for sequence sequence of successively decreasing, VxIt is the k of right singular vector
× n matrix, k<n.
It follows that the present embodiment does not execute SVD not instead of completely, k item before being iterated to calculate using power iteration method.
The acquisition method of surface optical field provided in an embodiment of the present invention, by obtaining each dough sheet net on object model surface
Multiple images of lattice in different perspectives, wherein the object model is the mould for formed after grid dividing by the surface of object
Type；For each patch grids, one image of acquisition, will as target image from multiple images of the patch grids
Remaining each image in the patch grids in addition to the target image is aligned with the target image, generates the dough sheet net
The image collection of lattice；Each image in each described image set is sampled, so that each in each described image set
The pixel of image is identical, and visual angle interval is equal；Described image set after the corresponding sampling of all patch grids is divided
Solution and compression, determine the surface optical field data of the object model.That is the present embodiment, surface optical field carry out compression process it
Before, sampled images are aligned, and then eliminate the influence of inaccurate geometry, to reduce surface optical field sampling and processing
To the dependence of precise geometrical model.
Fig. 6 is the flow example figure of the acquisition method of surface optical field provided by Embodiment 2 of the present invention.In above-described embodiment
On the basis of, what is involved is obtain from multiple images of the patch grids for each patch grids the present embodiment
Detailed process of one image as target image.As shown in fig. 6, above-mentioned S102 can specifically include：
The sum of S201, the energy for determining the corresponding all images of all patch grids.
In practical application, since brightness change is big etc., factors may use existing method that can not find target image.This
Embodiment solves this problem by finding optimized image in all images.It specifically, will be on object model surface
The color value of the corresponding all images of all patch gridsAs energy function, with all dough sheet nets of determination
The sum of the energy of the corresponding all images of lattice.
In a kind of example, the sum of the brightness of the corresponding all images of all patch grids is calculated, by the sum of brightness conduct
The sum of the energy of the corresponding all images of all patch grids.
In another example, the quality sum of the corresponding all images of all patch grids is calculated, quality sum is made
For the sum of the energy of the corresponding all images of all patch grids.
In another example, the quality sum of the corresponding all images of all patch grids is calculated, quality sum is made
For the sum of the energy of the corresponding all images of all patch grids.
In another example, the sum of brightness of the corresponding all images of all patch grids and quality sum are calculated,
Using the sum of brightness with quality sum as the sum of energy of image.
In a kind of possible implementation of the present embodiment, all patch grids pair can also be determined according to formula (3)
The sum of the energy for all images answered E (P)：
Wherein, the Pi fFor the color value of i-th image of patch grids f, the El(Pi f) it is the Pi fIt is corresponding bright
Angle value, the Eq(Pi f) it is the Pi fCorresponding sample quality, the f ' are the adjacent patch grids of the patch grids f, institute
It statesIt is described for the color value of the jth image of the patch grids f 'For the patch grids f and the face
Piece grid f ' shares the color difference on side.
Above-mentioned, El(Pi f), state Eq(Pi f)、It can be determined according to existing mode, the present embodiment is herein no longer
It repeats.
Optionally,
Assuming that diffusion should be captured under the conditions of the favorable luminance of not DE Specular Lighting, and in order to reach this purpose, meter
Calculate the luminance mean value of each patch gridsAnd varianceAbandon minimum 5% sample of average brightness, these samples may be
Do not have to be captured in the case where enough light.
Then it usesWithThe most probable luminance mean value of schema extraction and variance, define brightness El(Pi f) energy
Amount.
Since DE Specular Lighting may result inWithCompared to notable difference, therefore ElBe conducive to being shone for not no bloom
Bright patch grids.
Optionally,
It indicatesOriginal projection size.The present embodiment can choose the projection size of mass term, because it is
The group indicater of angular distance between camera position distance and camera view and triangle normal.
Optionally,
The present embodiment, P are the shared edge of patch grids f and patch grids f '.AndIt isThe RGB of middle p point
Information.In short, Es calculates the color difference on the shared side of two adjacent triangles.
S202, using each patch grids corresponding image when the sum of described energy minimum as the target of each patch grids
Image.
Specifically, can determine the sum of the energy of the corresponding all images of all patch grids according to aforesaid way.It connects
, the sum of energy minimum is enabled, can determine one group of image, this group of image is corresponded into the target figure as patch grids
Picture.
For example, enable above-mentioned formula (3) minimum, will at this time in formula (3) the corresponding image of each patch grids as each
The target image of patch grids.
The method of the present embodiment, when determining target image, it is contemplated that the color difference of the adjacent dough sheet of patch grids influences, because
This, so that the target image determined is more in line with actual needs.
Fig. 7 is the flow example figure of the acquisition method for the surface optical field that the embodiment of the present invention three provides.In above-described embodiment
On the basis of, the present embodiment what is involved is, by the patch grids in addition to the target image remaining each image and institute
State target image alignment.As shown in fig. 7, above-mentioned S102 can specifically include：
The similar energies value of S301, remaining for determining the patch grids each image and the target image.
Specifically, when being in the present embodiment aligned remaining each image of patch grids with target image, it can be by true
Determine remaining each image of patch grids and the similar energies value of target image, and makes similar energies value maximum to guarantee remaining each figure
As being aligned with target image.
The present embodiment to determine patch grids remaining each image and target image similar energies value concrete mode not
It is limited.
In a kind of example, above-mentioned S301 be can be according to formula (4), determine remaining each image of the patch grids with
The similar energies value E of the target imagef(Df,t)
Wherein, the DfFor the target image of the patch grids f, the Pi fFor the face of i-th image of patch grids f
Color value, the tiFor the translational movement of i-th image of the patch grids f.
It is in original image space there is 2D to shift t=(tx, ty) sampling model again.Due to needing
Similarity system design is carried out around mirror surface information, so mutual information measurement can be used as MI in the present embodiment.According to above-mentioned formula
(4) suitable D can be calculated by alternate search tf。
In another example, above-mentioned S301 be can be according to formula (5), determine remaining each image of the patch grids
With the similar energies value E of the target imagef(Df,t)
Wherein, the DfFor the target image of the patch grids f, the Pi fFor the face of i-th image of patch grids f
Color value, the tiFor the translational movement of i-th image of the patch grids f, the t0For preset value.
t0For avoiding zero offset and adjusting tiWeight, by distance limits tmaxInside wolfishly search for tiTo solve this
A problem.
Optionally, t0It can be (15,15), tmaxIt is set as 3 pixels.
S302, when the similar energies value maximum, determine the patch grids remaining each image and the target figure
As alignment.
According to above-mentioned steps, remaining each image of patch grids and the similar energies value E of the target image are determinedf(Df,
t).When similar energies value maximum, it can determine that remaining each image of patch grids is aligned with target image.
The present embodiment is by way of determining remaining each image of patch grids and the similar energies value of the target image
It is aligned remaining each image of patch grids with target image, and then improves the reliability and efficiency of alignment.
Fig. 8 is the structural schematic diagram of the acquisition device for the surface optical field that the embodiment of the present invention one provides.As shown in figure 8, this
The acquisition device 100 of the surface optical field of embodiment may include：
First obtains module 110, for obtaining multiple of each patch grids in different perspectives on object model surface
Image, wherein the object model is the model for formed after grid dividing by the surface of object；
Second obtains module 120, for being obtained from multiple images of the patch grids for each patch grids
Take an image as target image；
Alignment module 130, for by the patch grids in addition to the target image remaining each image with it is described
Target image alignment, generates the image collection of the patch grids；
Sampling module 140, for being sampled to each image in each described image set, so that each described image
The pixel of each image in set is identical, and visual angle interval is equal；
Compression module 150 is decomposed, for decomposing to the described image set after the corresponding sampling of all patch grids
And compression, determine the surface optical field data of the object model.
The acquisition device of the surface optical field of the embodiment of the present invention can be used for executing the technology of above-mentioned shown embodiment of the method
Scheme, it is similar that the realization principle and technical effect are similar, and details are not described herein again.
In a kind of possible implementation of the present embodiment, described second obtains module 120, for for each described
Patch grids, using the maximum image of diffusion color value in the corresponding each image of the patch grids as the mesh of the patch grids
Logo image.
Fig. 9 is the structural schematic diagram of the acquisition device of surface optical field provided by Embodiment 2 of the present invention.As shown in figure 9, institute
Stating the second acquisition module 120 includes：
First computing unit 121, for determining the sum of the energy of the corresponding all images of all patch grids；
Second determination unit 122, for using each patch grids corresponding image when the sum of described energy minimum as often
The target image of a patch grids.
In a kind of possible implementation of the present embodiment, first computing unit 121 is specifically used for according to formulaDetermine that the energy of every image of each patch grids is total
With E (P)；
Wherein, the Pi fFor the color value of i-th image of patch grids f, the El(Pi f) it is the Pi fIt is corresponding bright
Angle value, the Eq(Pi f) it is the Pi fCorresponding sample quality, the f ' are the adjacent patch grids of the patch grids f, institute
It statesIt is described for the color value of the jth image of the patch grids f 'For the patch grids f and the face
Piece grid f ' shares the color difference on side.
The acquisition device of the surface optical field of the embodiment of the present invention can be used for executing the technology of above-mentioned shown embodiment of the method
Scheme, it is similar that the realization principle and technical effect are similar, and details are not described herein again.
Figure 10 is the structural schematic diagram of the acquisition device for the surface optical field that the embodiment of the present invention three provides.In above-described embodiment
On the basis of, as described in Figure 10, the alignment module 130 further includes：
Second computing unit 131, remaining each image for determining the patch grids are similar to the target image
Energy value；
Second determination unit 132, for when the similar energies value maximum, determining remaining each figure of the patch grids
As being aligned with the target image.
In a kind of possible implementation of the present embodiment, second computing unit 131 is specifically used for according to formulaDetermine that remaining each image of the patch grids is similar to the target image
Energy value Ef(Df,t)；
Wherein, the DfFor the target image of the patch grids f, the Pi fFor the face of i-th image of patch grids f
Color value, the tiFor the translational movement of i-th image of the patch grids f.
In the alternatively possible implementation of the present embodiment, second computing unit 131, also particularly useful for basis
FormulaDetermine remaining each image of the patch grids with it is described
The similar energies value E of target imagef(Df,t)；
Wherein, the DfFor the target image of the patch grids f, the Pi fFor the face of i-th image of patch grids f
Color value, the tiFor the translational movement of i-th image of the patch grids f, the t0For preset value.
Optionally, the patch grids is triangle surface grid.
The acquisition device of the surface optical field of the embodiment of the present invention can be used for executing the technology of above-mentioned shown embodiment of the method
Scheme, it is similar that the realization principle and technical effect are similar, and details are not described herein again.
Figure 11 is the structural schematic diagram of electronic equipment provided in an embodiment of the present invention, as shown in figure 11, the electricity of the present embodiment
Sub- equipment 200 includes：
Memory 220, for storing computer program；
Processor 230, to realize the acquisition method of above-mentioned surface optical field, is realized for executing the computer program
Principle is similar with technical effect, and details are not described herein again.
Further, when at least part function of the acquisition method of surface optical field in the embodiment of the present invention passes through software reality
Now, the embodiment of the present invention also provides a kind of computer storage medium, and computer storage medium is above-mentioned to surface for being stored as
The computer software instructions of the acquisition of light field allow computer to execute above method reality when run on a computer
Apply the acquisition method of various possible surface optical fields in example.When loading on computers and executing the computer executed instructions,
It can entirely or partly generate according to process or function described in the embodiment of the present invention.The computer instruction, which can store, to be counted
In calculation machine storage medium, or from a computer storage medium to the transmission of another computer storage medium, the transmission can
To another web-site, computer, clothes in a manner of through wireless (such as cellular communication, infrared, short-distance wireless, microwave etc.)
Business device or data center are transmitted.The computer storage medium can be any usable medium that computer can access or
Person is comprising data storage devices such as one or more usable mediums integrated server, data centers.The usable medium can
To be magnetic medium, (for example, floppy disk, hard disk, tape), optical medium (for example, DVD) or semiconductor medium (such as SSD)
Deng.
Finally it should be noted that：The above embodiments are only used to illustrate the technical solution of the present invention., rather than its limitations；To the greatest extent
Present invention has been described in detail with reference to the aforementioned embodiments for pipe, those skilled in the art should understand that：Its according to
So be possible to modify the technical solutions described in the foregoing embodiments, or to some or all of the technical features into
Row equivalent replacement；And these are modified or replaceed, various embodiments of the present invention technology that it does not separate the essence of the corresponding technical solution
The range of scheme.
Claims (18)
1. a kind of acquisition method of surface optical field, which is characterized in that including：
Multiple images of each patch grids in different perspectives on object model surface are obtained, wherein the object model is will
The surface of object carries out the model formed after grid dividing；
For each patch grids, an image is obtained from multiple images of the patch grids as target image,
Remaining each image in the patch grids in addition to the target image is aligned with the target image, generates the dough sheet
The image collection of grid；
Each image in each described image set is sampled, so that the pixel of each image in each described image set
It is identical, and visual angle interval is equal；
Described image set after the corresponding sampling of all patch grids is decomposed and compressed, determines the object model
Surface optical field data.
2. the method according to claim 1, wherein described for each patch grids, from the dough sheet
An image is obtained in multiple images of grid as target image, including：
For each patch grids, using the maximum image of diffusion color value in the corresponding each image of the patch grids as
The target image of the patch grids.
3. the method according to claim 1, wherein described for each patch grids, from the dough sheet
An image is obtained in multiple images of grid as target image, including：
Determine the sum of the energy of the corresponding all images of all patch grids；
Using each patch grids corresponding image when the sum of described energy minimum as the target image of each patch grids.
4. according to the method described in claim 3, it is characterized in that, each patch grids of the determination corresponding each image
The sum of energy, including：
According to formulaDetermine every of each patch grids
The energy summation E (P) of image；
Wherein, describedIt is described for the color value of i-th image of patch grids fIt is describedCorresponding brightness value,
It is describedIt is describedCorresponding sample quality, the f ' is the adjacent patch grids of the patch grids f, described
It is described for the color value of the jth image of the patch grids f 'For the patch grids f and the dough sheet net
Lattice f ' shares the color difference on side.
5. method according to claim 1-4, which is characterized in that described to remove the mesh in the patch grids
Remaining each image except logo image is aligned with the target image, including：
Determine remaining each image of the patch grids and the similar energies value of the target image；
When the similar energies value maximum, determine that remaining each image of the patch grids is aligned with the target image.
6. according to the method described in claim 5, it is characterized in that, remaining each image of the determination patch grids and institute
The similar energies value of target image is stated, including：
According to formulaDetermine the patch grids remaining each image and the target
The similar energies value E of imagef(Df,t)；
Wherein, the DfIt is described for the target image of the patch grids fFor the color of i-th image of patch grids f
Value, the tiFor the translational movement of i-th image of the patch grids f.
7. according to the method described in claim 5, it is characterized in that, remaining each image of the determination patch grids and institute
The similar energies value of target image is stated, including：
According to formulaDetermine remaining each figure of the patch grids
As the similar energies value E with the target imagef(Df,t)；
Wherein, the DfIt is described for the target image of the patch grids fFor the color of i-th image of patch grids f
Value, the tiFor the translational movement of i-th image of the patch grids f, the t0For preset value.
8. method according to claim 1-4, which is characterized in that the patch grids is triangle surface net
Lattice.
9. a kind of acquisition device of surface optical field, which is characterized in that including：
First obtains module, for obtaining multiple images of each patch grids in different perspectives on object model surface,
Described in object model be that the surface of object is subjected to the model that is formed after grid dividing；
Second obtains module, for obtaining one from multiple images of the patch grids for each patch grids
Image is as target image；
Alignment module, for by the patch grids in addition to the target image remaining each image and the target image
Alignment, generates the image collection of the patch grids；
Sampling module, for being sampled to each image in each described image set, so that in each described image set
Each image pixel it is identical, and visual angle interval is equal；
Compression module is decomposed, for the described image set after the corresponding sampling of all patch grids to be decomposed and is compressed,
Determine the surface optical field data of the object model.
10. device according to claim 9, which is characterized in that
Described second obtains module, for will overflow in the corresponding each image of the patch grids for each patch grids
Penetrate target image of the maximum image of color value as the patch grids.
11. device according to claim 9, which is characterized in that described second, which obtains module, includes：
First computing unit, for determining the sum of the energy of the corresponding all images of all patch grids；
Second determination unit, for using each patch grids corresponding image when the sum of described energy minimum as each dough sheet net
The target image of lattice.
12. device according to claim 11, which is characterized in that first computing unit is specifically used for according to formulaDetermine that the energy of every image of each patch grids is total
With E (P)；
Wherein, describedIt is described for the color value of i-th image of patch grids fIt is describedCorresponding brightness value,
It is describedIt is describedCorresponding sample quality, the f ' is the adjacent patch grids of the patch grids f, described
It is described for the color value of the jth image of the patch grids f 'For the patch grids f and the dough sheet net
Lattice f ' shares the color difference on side.
13. according to the described in any item devices of claim 9-12, which is characterized in that the alignment module includes：
Second computing unit, for determining remaining each image of the patch grids and the similar energies value of the target image；
Second determination unit, for determining remaining each image and the institute of the patch grids when the similar energies value maximum
State target image alignment.
14. device according to claim 13, which is characterized in that
Second computing unit is specifically used for according to formulaDetermine the dough sheet
The similar energies value E of each image of remaining of grid and the target imagef(Df,t)；
Wherein, the DfIt is described for the target image of the patch grids fFor the color of i-th image of patch grids f
Value, the tiFor the translational movement of i-th image of the patch grids f.
15. device according to claim 13, which is characterized in that second computing unit, also particularly useful for according to public affairs
FormulaDetermine remaining each image and the mesh of the patch grids
The similar energies value E of logo imagef(Df,t)；
Wherein, the DfIt is described for the target image of the patch grids fFor the color of i-th image of patch grids f
Value, the tiFor the translational movement of i-th image of the patch grids f, the t0For preset value.
16. according to the described in any item devices of claim 9-12, which is characterized in that the patch grids is triangle surface net
Lattice.
17. a kind of electronic equipment, which is characterized in that including：
Memory, for storing computer program；
Processor, for executing the computer program, to realize such as surface optical field of any of claims 1-8
Acquisition method.
18. a kind of computer storage medium, which is characterized in that store computer program, the computer in the storage medium
Program realizes the acquisition method such as surface optical field of any of claims 1-8 when being executed.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810720340.XA CN108921908B (en) | 2018-07-03 | 2018-07-03 | Surface light field acquisition method and device and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810720340.XA CN108921908B (en) | 2018-07-03 | 2018-07-03 | Surface light field acquisition method and device and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108921908A true CN108921908A (en) | 2018-11-30 |
| CN108921908B CN108921908B (en) | 2020-07-28 |
Family
ID=64425343
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810720340.XA Active CN108921908B (en) | 2018-07-03 | 2018-07-03 | Surface light field acquisition method and device and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108921908B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113570620A (en) * | 2021-07-14 | 2021-10-29 | 北京达佳互联信息技术有限公司 | Light field data acquisition method and device, electronic equipment and storage medium |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101706830A (en) * | 2009-11-12 | 2010-05-12 | 中国人民解放军国防科学技术大学 | Method for reconstructing model after drilling surface grid model of rigid object |
| CN102306396A (en) * | 2011-09-15 | 2012-01-04 | 山东大学 | Three-dimensional entity model surface finite element mesh automatic generation method |
| CN104392477A (en) * | 2014-12-11 | 2015-03-04 | 大连大学 | Triangular mesh curve surface recovery post-treatment method based on removal of overlapped patches with included angles |
| CN104778755A (en) * | 2015-03-27 | 2015-07-15 | 浙江理工大学 | Region-division-based three-dimensional reconstruction method for texture image |
| US20170004750A1 (en) * | 2012-08-04 | 2017-01-05 | Paul Lapstun | Light Field Display Device and Method |
| CN106373179A (en) * | 2016-08-29 | 2017-02-01 | 北京像素软件科技股份有限公司 | Illumination scene rendering method |
| CN106683068A (en) * | 2015-11-04 | 2017-05-17 | 北京文博远大数字技术有限公司 | Three-dimensional digital image acquisition method and equipment thereof |
| CN107918926A (en) * | 2017-11-22 | 2018-04-17 | 徐李添 | A kind of digital image magnification method based on quadrangle and triangle surface |
| US20180139436A1 (en) * | 2016-11-11 | 2018-05-17 | Disney Enterprises, Inc. | Object reconstruction from dense light fields via depth from gradients |
-
2018
- 2018-07-03 CN CN201810720340.XA patent/CN108921908B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101706830A (en) * | 2009-11-12 | 2010-05-12 | 中国人民解放军国防科学技术大学 | Method for reconstructing model after drilling surface grid model of rigid object |
| CN102306396A (en) * | 2011-09-15 | 2012-01-04 | 山东大学 | Three-dimensional entity model surface finite element mesh automatic generation method |
| US20170004750A1 (en) * | 2012-08-04 | 2017-01-05 | Paul Lapstun | Light Field Display Device and Method |
| CN104392477A (en) * | 2014-12-11 | 2015-03-04 | 大连大学 | Triangular mesh curve surface recovery post-treatment method based on removal of overlapped patches with included angles |
| CN104778755A (en) * | 2015-03-27 | 2015-07-15 | 浙江理工大学 | Region-division-based three-dimensional reconstruction method for texture image |
| CN106683068A (en) * | 2015-11-04 | 2017-05-17 | 北京文博远大数字技术有限公司 | Three-dimensional digital image acquisition method and equipment thereof |
| CN106373179A (en) * | 2016-08-29 | 2017-02-01 | 北京像素软件科技股份有限公司 | Illumination scene rendering method |
| US20180139436A1 (en) * | 2016-11-11 | 2018-05-17 | Disney Enterprises, Inc. | Object reconstruction from dense light fields via depth from gradients |
| CN107918926A (en) * | 2017-11-22 | 2018-04-17 | 徐李添 | A kind of digital image magnification method based on quadrangle and triangle surface |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113570620A (en) * | 2021-07-14 | 2021-10-29 | 北京达佳互联信息技术有限公司 | Light field data acquisition method and device, electronic equipment and storage medium |
| CN113570620B (en) * | 2021-07-14 | 2024-03-05 | 北京达佳互联信息技术有限公司 | Light field data acquisition method and device, electronic equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108921908B (en) | 2020-07-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108509848B (en) | The real-time detection method and system of three-dimension object | |
| CN111598998B (en) | Three-dimensional virtual model reconstruction method, three-dimensional virtual model reconstruction device, computer equipment and storage medium | |
| CN115082639B (en) | Image generation method, device, electronic equipment and storage medium | |
| Egan et al. | Frequency analysis and sheared filtering for shadow light fields of complex occluders | |
| JP4880090B2 (en) | Image generating apparatus and method for super-resolution of 3D texture | |
| Li et al. | Detail-preserving and content-aware variational multi-view stereo reconstruction | |
| US20240046557A1 (en) | Method, device, and non-transitory computer-readable storage medium for reconstructing a three-dimensional model | |
| CN113220251B (en) | Object display method, device, electronic equipment and storage medium | |
| CN113643414B (en) | Three-dimensional image generation method and device, electronic equipment and storage medium | |
| WO2024007478A1 (en) | Three-dimensional human body modeling data collection and reconstruction method and system based on single mobile phone | |
| Jeon et al. | Texture map generation for 3D reconstructed scenes | |
| CN111612882A (en) | Image processing method, image processing device, computer storage medium and electronic equipment | |
| CN115797555A (en) | Human body real-time three-dimensional reconstruction method based on depth camera | |
| CN114863061A (en) | Three-dimensional reconstruction method and system for remote monitoring medical image processing | |
| Tzevanidis et al. | From multiple views to textured 3d meshes: a gpu-powered approach | |
| CN114359445A (en) | Training data generation method and device for human body bone joint point extraction model | |
| CN109166176B (en) | Three-dimensional face image generation method and device | |
| Pintus et al. | Techniques for seamless color registration and mapping on dense 3D models | |
| CN116188660B (en) | Point cloud data processing method and related device based on stream rendering | |
| CN108921908A (en) | Acquisition method, device and the electronic equipment of surface optical field | |
| Hapák et al. | Real-time 4D reconstruction of human motion | |
| Xing et al. | Online illumination estimation of outdoor scenes based on videos containing no shadow area | |
| Yılmaz et al. | Illumination-guided inverse rendering benchmark: Learning real objects with few cameras | |
| CN107451540B (en) | Compressible 3D identification method | |
| Melendez et al. | Relightable Buildings from Images. |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |