CN116612262B - Automatic grid adjustment method, system, equipment and medium aligned with reference photo - Google Patents
Automatic grid adjustment method, system, equipment and medium aligned with reference photo Download PDFInfo
- Publication number
- CN116612262B CN116612262B CN202310888005.1A CN202310888005A CN116612262B CN 116612262 B CN116612262 B CN 116612262B CN 202310888005 A CN202310888005 A CN 202310888005A CN 116612262 B CN116612262 B CN 116612262B
- Authority
- CN
- China
- Prior art keywords
- dimensional
- grid
- photo
- offset
- adjustment
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 41
- 238000013507 mapping Methods 0.000 claims abstract description 27
- 238000012216 screening Methods 0.000 claims abstract description 16
- 230000006870 function Effects 0.000 claims description 27
- 238000003860 storage Methods 0.000 claims description 9
- 238000006243 chemical reaction Methods 0.000 claims description 5
- 230000000007 visual effect Effects 0.000 claims description 3
- 238000005516 engineering process Methods 0.000 abstract description 8
- 238000004590 computer program Methods 0.000 description 14
- 239000011159 matrix material Substances 0.000 description 9
- 238000010586 diagram Methods 0.000 description 8
- 238000004364 calculation method Methods 0.000 description 7
- 230000008569 process Effects 0.000 description 7
- 238000012545 processing Methods 0.000 description 6
- 230000003993 interaction Effects 0.000 description 4
- 230000000694 effects Effects 0.000 description 3
- 238000004519 manufacturing process Methods 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 210000000746 body region Anatomy 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000008034 disappearance Effects 0.000 description 1
- 238000006073 displacement reaction Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000000338 in vitro Methods 0.000 description 1
- 238000001727 in vivo Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/20—Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
- G06T17/20—Finite element generation, e.g. wire-frame surface description, tesselation
-
- G06T3/06—
-
- G06T3/18—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/13—Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30196—Human being; Person
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2219/00—Indexing scheme for manipulating 3D models or images for computer graphics
- G06T2219/20—Indexing scheme for editing of 3D models
- G06T2219/2021—Shape modification
Abstract
The application discloses a grid automatic adjustment method, a system, equipment and a medium aligned with a reference photo, wherein the method comprises the following steps: acquiring a three-dimensional human body model generated by a reference photo and camera parameters corresponding to the reference photo; converting the reference picture into a binary image, extracting human body contour information, and obtaining a two-dimensional picture contour; establishing a mapping relation from a three-dimensional space to a two-dimensional photo according to the camera parameters; screening three-dimensional human edge grid data corresponding to the two-dimensional photo outline according to the mapping relation to obtain grids needing fine adjustment; and constructing an objective function according to the grid to be trimmed, and realizing automatic grid adjustment by optimizing the objective function. The application solves the problems of errors and data inconsistency caused by manual operation intervention in the traditional fine tuning technology, and greatly improves the fine tuning efficiency while ensuring the precision; and original topological structure and grid characteristic information are reserved on grid fine tuning, so that grid quality is ensured.
Description
Technical Field
The application relates to the technical field of grid deformation, in particular to a method, a system, equipment and a medium for automatically adjusting grids aligned with a reference photo.
Background
Grid fine tuning, namely grid deformation, refers to a geometric processing technology for modifying the shape of a geometric model through constraint, and has wide application in the fields of industrial design, video entertainment and the like. Through geometric deformation, a designer can improve modeling efficiency and model reality by utilizing the existing three-dimensional model information under the condition of not changing the topological connection relation of the model.
Problems with the prior art include:
(1) The traditional grid deformation technology is performed in a large number of manual interaction modes, and is low in efficiency due to long-time manual interaction operation by referring to a specific photo outline.
(2) The traditional grid deformation technology is carried out according to manual operation, so that deformation results are different, and the standards in industries such as industry, film and television are difficult to reach.
(3) Under the condition of dense grids, the conventional grid deformation is easy to cause problems of selfing, local collapse, shape distortion and the like.
Disclosure of Invention
In order to overcome the above-mentioned shortcomings of the prior art, the present application provides a method, system, device and medium for automatically adjusting grid alignment with reference pictures, which are used for solving at least one of the above-mentioned technical problems.
According to an aspect of the present disclosure, there is provided a grid automatic adjustment method aligned with a reference photo, including:
acquiring a three-dimensional human body model generated by a reference photo and camera parameters corresponding to the reference photo;
converting the reference picture into a binary image, extracting human body contour information, and obtaining a two-dimensional picture contour;
establishing a mapping relation from a three-dimensional space to a two-dimensional photo according to the camera parameters;
screening three-dimensional human edge grid data corresponding to the two-dimensional photo outline according to the mapping relation to obtain grids needing fine adjustment;
and constructing an objective function according to the grid to be trimmed, and realizing automatic grid adjustment by optimizing the objective function.
According to the technical scheme, the mapping relation between the two-dimensional space and the three-dimensional space is obtained by using the camera parameters, the three-dimensional grid outline information is stored in the two-dimensional space, grid data corresponding to the two-dimensional information are screened under the condition that the topological structure is not changed, and the three-dimensional grid is finely adjusted by using an optimization method, so that the human body grid is close to a photo at a specific observation angle, and the effect of automatically finely adjusting according to requirements is achieved. The technical scheme solves the problems of errors and data inconsistency caused by manual operation intervention in the traditional fine adjustment technology, and greatly improves fine adjustment efficiency while guaranteeing accuracy; and original topological structure and grid characteristic information are reserved on grid fine tuning, so that grid quality is ensured.
As a further technical solution, the method further includes:
and carrying out convolution operation on each pixel in the binary image by adopting a specific convolution check to obtain a photo contour with vector information.
Specifically, a specific convolution is used for checking each pixel in the binary image to carry out convolution operation, so that a contour image with vector information is obtained, and the external and internal areas of a human body in two-dimensional information are distinguished.
As a further technical solution, the specific convolution kernel is:
convolution kernel1:
convolution kernel2:。
specifically, vector information is obtained by utilizing specific convolution kernel calculation, in-vitro and in-vivo regions of a human body are accurately distinguished, and the accuracy of screening the corresponding regions can be greatly improved.
As a further technical solution, the method further includes:
the three-dimensional manikin generated from the reference photograph is regional divided.
According to the technical scheme, the three-dimensional human body model data are divided into the regions, so that a specific region can be selected for deformation, and the influence of other regions near the target point in edge matching can be effectively eliminated.
As a further technical solution, the method further includes:
after a grid to be finely tuned is obtained, according to the two-dimensional point P2d of the edge point and the obtained three-dimensional point P3d, combining the mapping relation to obtain a target position Ptarget of the two-dimensional point P2d in a three-dimensional coordinate system, and calculating offset=Ptarget-P3 d;
setting the radius of a region to be finely adjusted according to the offset to obtain all points needing to be changed in the radius of the region;
and (3) coding and storing all points needing to be changed, calculating a double-tone sum function, using double-harmonic deformation, taking double Laplacian of each space coordinate function as 0 as constraint, using offset information as a deformation field, solving the deformation field offset ', and then calculating a deformation position Ptarget ' =P3d+offset ', so as to obtain a final fine-tuning grid result.
As a further technical solution, the method further includes:
and obtaining the corresponding relation between the photo pixel point and the three-dimensional point under the fixed visual angle according to the camera parameters.
According to an aspect of the present specification, there is provided an automatic mesh adjustment system aligned with a reference photograph, including:
the input module is used for acquiring a three-dimensional human body model generated by the reference picture and camera parameters corresponding to the reference picture;
the conversion module is used for converting the reference picture into a binary image, extracting human body contour information and obtaining a two-dimensional picture contour;
the mapping module is used for establishing a mapping relation from the three-dimensional space to the two-dimensional photo according to the camera parameters;
the screening module is used for screening three-dimensional human body edge grid data corresponding to the two-dimensional photo outline according to the mapping relation to obtain grids needing fine adjustment;
and the adjusting module is used for constructing an objective function according to the grid to be finely adjusted, and realizing automatic grid adjustment through optimizing the objective function.
According to an aspect of the present specification, there is provided an electronic apparatus including:
a processor; and
and a memory having executable code stored thereon, which when executed by the processor, causes the processor to perform the steps of the grid auto-adjustment method aligned with a reference photo.
According to an aspect of the present description, there is provided a computer-readable storage medium having executable code stored thereon, wherein the executable code, when executed by a processor, implements the steps of the grid automatic adjustment method for alignment with a reference photo.
Compared with the prior art, the application has the beneficial effects that:
1. efficiency of fine tuning is improved: compared with manual interaction operation, the automatic fine adjustment scheme adopted by the application can completely omit the manual interaction process, and greatly improve the fine adjustment work efficiency.
2. Improving the precision of fine adjustment: the automatic fine tuning standard adopted by the application is based on computer graphics, can achieve higher precision in the aspect of contour information extraction, ensures that the contour information under a specific angle is consistent with the required two-dimensional contour, and enables the fine tuning result to be more close to the production requirement.
3. Improving consistency of fine tuning results: the automatic fine tuning technology adopted by the application has accurate and fixed calculation method, can eliminate errors generated by the traditional manual operation and ensures the consistency of the automatic fine tuning result because the manual factors almost cause the difference of fine tuning individuals.
4. Improving the quality of the grid after fine adjustment: the grid fine adjustment technology at the present stage is various, and under the condition of dense grids, the phenomena of grid disorder, deformity, feature disappearance and the like are easy to occur.
Drawings
Fig. 1 is a flowchart of a method for automatically adjusting a grid aligned with a reference photo according to an embodiment of the present application.
Fig. 2 is a human body mesh according to an embodiment of the present application.
Fig. 3 is a grid view under a particular camera according to an embodiment of the application.
Fig. 4 is a target binary image according to an embodiment of the present application.
Fig. 5 is a target profile according to an embodiment of the present application.
Fig. 6 is a diagram of a human body mesh after fine tuning according to an embodiment of the present application.
Fig. 7 is a diagram showing a fine tuning effect of a partial region according to an embodiment of the present application.
Detailed Description
The following description of the embodiments of the present application will be made more apparent and fully hereinafter with reference to the accompanying drawings, in which some, but not all embodiments of the application are shown. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to fall within the scope of the application.
The application provides an automatic grid adjustment method aligned with a reference photo, which utilizes camera parameters to acquire the mapping relation between a two-dimensional space and a three-dimensional space, stores three-dimensional grid outline information in the two-dimensional space, screens grid data corresponding to the two-dimensional information under the condition of not changing a topological structure, and uses an optimization method to finely adjust the three-dimensional grid, so that the human grid is close to the photo at a specific observation angle, and the effect of automatically finely adjusting according to requirements is achieved.
The application solves the problems of errors and data inconsistency caused by manual operation intervention in the traditional fine tuning technology, and greatly improves the fine tuning efficiency while ensuring the precision; and original topological structure and grid characteristic information are reserved on grid fine tuning, so that grid quality is ensured.
As shown in fig. 1, the method includes:
step 1, acquiring a three-dimensional human body model generated by a reference photo and camera parameters corresponding to the reference photo;
step 2, converting the reference picture into a binary image, extracting human body contour information and obtaining a two-dimensional picture contour;
step 3, establishing a mapping relation from a three-dimensional space to a two-dimensional photo according to the camera parameters;
step 4, screening three-dimensional human edge grid data corresponding to the two-dimensional photo outline according to the mapping relation to obtain grids needing fine adjustment;
and 5, constructing an objective function according to the grid to be trimmed, and realizing automatic grid adjustment by optimizing the objective function.
In particular, the three-dimensional manikin generated from the reference photographs has a specific triangular mesh topology. As shown in fig. 2 and 3.
In step 2, in the process of extracting the human body contour, a rectangular frame containing the human body in the photo is extracted first, and then the human body region is extracted, as shown in fig. 4. After the binary image is obtained, the convolution calculation is carried out to the normal information to draw the outline information image, and the accuracy of screening the corresponding area can be greatly improved through the information.
Specifically, a specific convolution is used for checking each pixel in the binary image to carry out convolution operation, so that a contour image with vector information is obtained, and the external and internal areas of a human body in two-dimensional information are distinguished.
The specific convolution kernel is:
convolution kernel1:
convolution kernel2:。
the convolution process is specifically as follows: the pixel point value of the binary image is 0 (black) or 255 (white), the (x, y) pixels are sequentially selected, the pixel is taken as a convolution center, a 3*3 area is taken as a convolution area and is input, convolution calculation is carried out to obtain a 5*5 matrix, and all element values of the matrix are added to obtain a result: the kernel1 result is x_result, and the kernel2 result is y_result; the (x_result+255)/2, and the (y_result+255)/2 are respectively stored in two color channels with coordinates of (x, y) pixels, and as known from the calculation process, if the convolution result of the completely black convolution area is 0, the value of the stored color channel is 127 (gray), so as to obtain the target contour information graph. As shown in fig. 5.
In step 3, the camera parameters include camera intrinsic parameters and extrinsic parameters. And obtaining the corresponding relation between the photo pixel points and the three-dimensional points at a fixed visual angle according to the camera parameters, and preparing for subsequent screening of the grid area.
The specific implementation mode is as follows:
obtaining a View matrix view= [ camera. X, -camera. Y, -camera. Z, 1], camera internal parameters fx, fy, farPlane, nearest plane, screen center point coordinates (Cx, cy), photo width, photo height, camera matrix prj= [2 x/width, 0, 0, 0, 0, 0, 2 x fy/height, 0, 0, 1-2 x Cx/width, 2 x Cy/height-1, (farplane+nearest plane)/(nearest plane-nearest plane) ];
assuming that the coordinates of the point p in the three-dimensional space are (Px, py, pz), the 4D coordinates v4_screen=matrix [ Px, py, pz, 1 ]. Times.view. Prj, v4_screen=v4_screen/v4_screen. W (the formula conversion here is to represent the three-dimensional coordinates with homogeneous coordinates), the two-dimensional screen coordinates can be solved: x= (v4_screen.x+1) 0.5 x with, y= (1- (v4_screen.y+1) 0.5) height, i.e. the mapping of three-dimensional space to two-dimensional photo is achieved.
Further, the 4D coordinates refer to homogeneous coordinates of three-dimensional coordinates. In geometric transformation, matrix operation is used for accelerating operation speed and simplifying calculation, and a fourth component w is introduced for unified calculation and is called a scale factor, so that three components (x, y, z) of a three-dimensional coordinate are expressed as (x/w, y/x, z/w) by homogeneous coordinates. When w=0, it does not represent a specific coordinate position, but a vector having a size and a direction.
In step 4, the two-dimensional coordinate points of the photo contour map are used for screening out corresponding points or grids in the three-dimensional space through a two-dimensional/three-dimensional mapping relation.
The specific process of screening is as follows:
when the grid data is read, a data structure of a vertex and a vertex, an edge and a triangle adjacent to the vertex is constructed. Under the view angle of a camera in a three-dimensional space, calculating edge point data, calculating point coordinates (x, y) in the two-dimensional space according to a mapping relation, calculating a capturing area with the height and the width of 12 pixels by taking (x, y) as the center, and obtaining contour points in a contour map with the nearest straight line distance in the area to finish screening and matching.
Here, the edge point refers to a point of an edge of the three-dimensional object in a specific view angle.
The acquisition of the contour point in the contour map having the closest straight line distance in the region refers to the straight line distance between the contour point (x, y) and the center point (x 0, y 0) in the two-dimensional imageThe point at which the value of (2) is the smallest.
In step 4, the three-dimensional data is divided into areas (under the condition that the topological structure is unchanged, the serial numbers of points of the specific areas are not changed, marks are marked to achieve the purpose of distinguishing), the specific areas can be selected for deformation, and the influence of other areas near the target point in edge matching can be effectively eliminated.
The three-dimensional human body contour refers to an edge contour of the three-dimensional human body data projected in a two-dimensional coordinate system.
In step 5, after the grid to be fine-tuned is obtained in step 4, the target position Ptarget of the two-dimensional point P2d in the three-dimensional coordinate system is obtained according to the mapping relation in step 2, the offset=ptarget-P3 d is obtained, the radius of the area to be fine-tuned is set, and all the points to be varied in the fixed radius are obtained.
And (5) encoding and storing the grid by using Eigen, calculating a double-harmonic function, and finishing double-harmonic deformation.
The mesh is represented by a matrix V, which is a matrix of n x 3, each row storing xyz coordinates of one vertex. The matrix F represents triangle positional relationships, and each row stores a row index of one triangle vertex in V. Whereby the method stores mesh information, i.e. the encoding process.
Ptarget is a 3D point set calculated from 2D edge points, i.e. the target position that should eventually be corrected.
Ptarget is a point of a target grid position, double harmonic deformation is used, double Laplacian of each space coordinate function is used as constraint, offset information is used as deformation field, namely Ptarget=P3d+offset, constraint delta is carried out, deformation field offset 'is solved first, ptarget' =P3d+offset 'is calculated, and the final fine-tuning grid result (grid formed by Ptarget') is obtained after the deformation position is recovered. As shown in fig. 6 and 7.
The biharmonic function is a solution to the double Laplace equation, and for displacement, the energy can be expressed as。
The application also provides an automatic grid adjustment system aligned with the reference photo, which comprises:
the input module is used for acquiring a three-dimensional human body model generated by the reference picture and camera parameters corresponding to the reference picture;
the conversion module is used for converting the reference picture into a binary image, extracting human body contour information and obtaining a two-dimensional picture contour;
the mapping module is used for establishing a mapping relation from the three-dimensional space to the two-dimensional photo according to the camera parameters;
the screening module is used for screening three-dimensional human body edge grid data corresponding to the two-dimensional photo outline according to the mapping relation to obtain grids needing fine adjustment;
and the adjusting module is used for constructing an objective function according to the grid to be finely adjusted, and realizing automatic grid adjustment through optimizing the objective function.
The conversion module is further used for carrying out convolution operation on each pixel in the binary image by adopting a specific convolution check to obtain a photo outline with vector information.
The specific convolution kernel is:
convolution kernel1:
convolution kernel2:。
the adjusting module is further configured to perform the following steps:
after a grid to be finely tuned is obtained, according to the two-dimensional point P2d of the edge point and the obtained three-dimensional point P3d, combining the mapping relation to obtain a target position Ptarget of the two-dimensional point P2d in a three-dimensional coordinate system, and calculating offset=Ptarget-P3 d;
setting the radius of a region to be finely adjusted according to the offset to obtain all points needing to be changed in the radius of the region;
and (3) coding and storing all points needing to be changed, calculating a double-tone sum function, using double-harmonic deformation, taking a double Laplacian of each space coordinate function as 0 as constraint, using offset information as a deformation field, solving the deformation field offset ', and then calculating a deformation position Ptarget ' =P3d+offset ', and recovering the deformation position to obtain a final fine-tuning grid result.
The system may be implemented using embodiments of the method of the present application.
The application also provides electronic equipment which can be an industrial personal computer, a server or a computer terminal. The electronic device comprises a processor, a memory, and a computer program stored on the memory and executable by the processor, wherein the computer program, when executed by the processor, implements the steps of the grid automatic adjustment method aligned with a reference photo.
The electronic device comprises a processor, a memory, and a network interface connected via a system bus, wherein the memory may be
Including non-volatile storage media and internal memory. The non-volatile storage medium may store an operating system and a computer program. The method comprises
The computer program comprises program instructions which, when executed, cause the processor to perform the steps of any of the grid auto-adjustment methods for alignment with the reference picture.
The processor is used to provide computing and control capabilities to support the operation of the entire electronic device. The internal memory provides an environment for the execution of a computer program in a non-volatile storage medium that, when executed by a processor, causes the processor to perform the steps of any of the grid auto-adjustment methods that align with a reference picture.
The network interface is used for network communication such as transmitting assigned tasks and the like. It should be appreciated that the processor may be a central processing unit (CentralProcessingUnit, CPU), but may also be other general purpose processors, digital signal processors (DigitalSignalProcessor, DSP), application specific integrated circuits (ApplicationSpecificIntegratedCircuit, ASIC), field programmable gate arrays (Field-ProgrammableGateArray, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like. Wherein the general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The present application also provides a computer readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the grid automatic adjustment method aligned with a reference photo.
In the description of the present specification, reference to the terms "one embodiment," "certain embodiments," "illustrative embodiments," "examples," "specific examples," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the application. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiments or examples. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present application, and are not limiting; although the application has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some or all of the technical features thereof can be replaced with equivalents; these modifications or substitutions do not depart from the essence of the corresponding technical solutions from the technical solutions of the embodiments of the present application.
Claims (8)
1. A method for automatically adjusting a grid aligned with a reference photograph, comprising:
acquiring a three-dimensional human body model generated by a reference photo and camera parameters corresponding to the reference photo;
converting the reference picture into a binary image, extracting human body contour information, and obtaining a two-dimensional picture contour;
establishing a mapping relation from a three-dimensional space to a two-dimensional photo according to the camera parameters;
screening three-dimensional human edge grid data corresponding to the two-dimensional photo outline according to the mapping relation to obtain grids needing fine adjustment;
constructing an objective function according to the grid to be finely tuned, and realizing automatic grid adjustment by optimizing the objective function; further comprises: after a grid to be finely tuned is obtained, according to the two-dimensional point P2d of the edge point and the obtained three-dimensional point P3d, combining the mapping relation to obtain a target position Ptarget of the two-dimensional point P2d in a three-dimensional coordinate system, and calculating offset=Ptarget-P3 d;
setting the radius of a region to be finely adjusted according to the offset to obtain all points needing to be changed in the radius of the region;
and (3) coding and storing all points needing to be changed, calculating a double-tone sum function, using double-harmonic deformation, taking double Laplacian of each space coordinate function as 0 as constraint, using offset information as a deformation field, solving the deformation field offset ', and then calculating a deformation position Ptarget ' =P3d+offset ', so as to obtain a final fine-tuning grid result.
2. The method for automatically adjusting a grid aligned with a reference photograph according to claim 1, further comprising:
and carrying out convolution operation on each pixel in the binary image by adopting a specific convolution check to obtain a photo contour with vector information.
3. The method of automatic grid adjustment aligned to a reference photo of claim 2, wherein the specific convolution kernel is:
convolution kernel1:
convolution kernel2:
4. the method for automatically adjusting a grid aligned with a reference photograph according to claim 1, further comprising:
the three-dimensional manikin generated from the reference photograph is regional divided.
5. The method for automatically adjusting a grid aligned with a reference photograph according to claim 1, further comprising:
and obtaining the corresponding relation between the photo pixel point and the three-dimensional point under the fixed visual angle according to the camera parameters.
6. An automatic grid adjustment system for alignment with a reference photograph, comprising:
the input module is used for acquiring a three-dimensional human body model generated by the reference picture and camera parameters corresponding to the reference picture;
the conversion module is used for converting the reference picture into a binary image, extracting human body contour information and obtaining a two-dimensional picture contour;
the mapping module is used for establishing a mapping relation from the three-dimensional space to the two-dimensional photo according to the camera parameters;
the screening module is used for screening three-dimensional human body edge grid data corresponding to the two-dimensional photo outline according to the mapping relation to obtain grids needing fine adjustment;
the adjusting module is used for constructing an objective function according to the grid to be finely adjusted, and realizing automatic grid adjustment through optimizing the objective function;
the adjusting module is further configured to calculate an offset=ptarget-P3 d according to the two-dimensional point P2d of the edge point and the obtained three-dimensional point P3d, and by combining the mapping relationship, calculate a target position Ptarget of the two-dimensional point P2d in the three-dimensional coordinate system after the grid to be fine-tuned is obtained;
setting the radius of a region to be finely adjusted according to the offset to obtain all points needing to be changed in the radius of the region;
and (3) coding and storing all points needing to be changed, calculating a double-tone sum function, using double-harmonic deformation, taking double Laplacian of each space coordinate function as 0 as constraint, using offset information as a deformation field, solving the deformation field offset ', and then calculating a deformation position Ptarget ' =P3d+offset ', so as to obtain a final fine-tuning grid result.
7. An electronic device, comprising:
a processor; and
a memory having executable code stored thereon, which when executed by the processor causes the processor to perform the steps of the grid auto-adjustment method for alignment with a reference photo as claimed in any one of claims 1 to 5.
8. A computer readable storage medium having stored thereon executable code, wherein the executable code when executed by a processor performs the steps of the grid auto-adjustment method for alignment with a reference photo of any of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310888005.1A CN116612262B (en) | 2023-07-19 | 2023-07-19 | Automatic grid adjustment method, system, equipment and medium aligned with reference photo |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310888005.1A CN116612262B (en) | 2023-07-19 | 2023-07-19 | Automatic grid adjustment method, system, equipment and medium aligned with reference photo |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116612262A CN116612262A (en) | 2023-08-18 |
| CN116612262B true CN116612262B (en) | 2023-09-29 |
Family
ID=87675050
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310888005.1A Active CN116612262B (en) | 2023-07-19 | 2023-07-19 | Automatic grid adjustment method, system, equipment and medium aligned with reference photo |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116612262B (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101853523A (en) * | 2010-05-18 | 2010-10-06 | 南京大学 | Method for adopting rough drawings to establish three-dimensional human face molds |
| JP2013089123A (en) * | 2011-10-20 | 2013-05-13 | National Institute Of Information & Communication Technology | Generation method, generation program, and generation system of individual model data |
| CN110443885A (en) * | 2019-07-18 | 2019-11-12 | 西北工业大学 | Three-dimensional number of people face model reconstruction method based on random facial image |
| CN114080628A (en) * | 2018-11-27 | 2022-02-22 | 斯纳普公司 | Textured mesh construction |
| CN114596198A (en) * | 2020-12-03 | 2022-06-07 | 北京新氧科技有限公司 | Image processing method, device, equipment and storage medium |
| CN114969856A (en) * | 2021-02-20 | 2022-08-30 | 杭州西璧网络科技有限公司 | Method for adjusting curved surface edge recognition and editing range implementation of grid model |
| CN115049799A (en) * | 2022-06-14 | 2022-09-13 | 北京百度网讯科技有限公司 | Method and device for generating 3D model and virtual image |
| CN115965765A (en) * | 2022-12-02 | 2023-04-14 | 东南大学 | Human motion capture method in deformable scene based on neural deformation |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10832472B2 (en) * | 2018-10-22 | 2020-11-10 | The Hong Kong Polytechnic University | Method and/or system for reconstructing from images a personalized 3D human body model and thereof |
| US11734890B2 (en) * | 2020-08-17 | 2023-08-22 | Nvidia Corporation | Three-dimensional model recovery from two-dimensional images |
| US11562536B2 (en) * | 2021-03-15 | 2023-01-24 | Tencent America LLC | Methods and systems for personalized 3D head model deformation |
-
2023
- 2023-07-19 CN CN202310888005.1A patent/CN116612262B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101853523A (en) * | 2010-05-18 | 2010-10-06 | 南京大学 | Method for adopting rough drawings to establish three-dimensional human face molds |
| JP2013089123A (en) * | 2011-10-20 | 2013-05-13 | National Institute Of Information & Communication Technology | Generation method, generation program, and generation system of individual model data |
| CN114080628A (en) * | 2018-11-27 | 2022-02-22 | 斯纳普公司 | Textured mesh construction |
| CN110443885A (en) * | 2019-07-18 | 2019-11-12 | 西北工业大学 | Three-dimensional number of people face model reconstruction method based on random facial image |
| CN114596198A (en) * | 2020-12-03 | 2022-06-07 | 北京新氧科技有限公司 | Image processing method, device, equipment and storage medium |
| CN114969856A (en) * | 2021-02-20 | 2022-08-30 | 杭州西璧网络科技有限公司 | Method for adjusting curved surface edge recognition and editing range implementation of grid model |
| CN115049799A (en) * | 2022-06-14 | 2022-09-13 | 北京百度网讯科技有限公司 | Method and device for generating 3D model and virtual image |
| CN115965765A (en) * | 2022-12-02 | 2023-04-14 | 东南大学 | Human motion capture method in deformable scene based on neural deformation |
Non-Patent Citations (2)
| Title |
|---|
| Jihyun Lee,etc.Pop-Out Motion: 3D-Aware Image Deformation via Learning the Shape Laplacian.《Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)》.2022,18532-18541. * |
| 分阶段的三维人体参数化重建;朱睿鼎;《中国优秀硕士学位论文全文数据库 信息科技辑》;I138-2897 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116612262A (en) | 2023-08-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110363858B (en) | Three-dimensional face reconstruction method and system | |
| CN109872397B (en) | Three-dimensional reconstruction method of airplane parts based on multi-view stereo vision | |
| EP2678824B1 (en) | Determining model parameters based on transforming a model of an object | |
| US9652849B2 (en) | Techniques for rapid stereo reconstruction from images | |
| JP5294343B2 (en) | Image alignment processing device, area expansion processing device, and image quality improvement processing device | |
| JP2009230537A (en) | Image processor, image processing program, image processing method, and electronic equipment | |
| CN111242848B (en) | Binocular camera image suture line splicing method and system based on regional feature registration | |
| JP2011198330A (en) | Method and program for collation in three-dimensional registration | |
| CN114708375B (en) | Texture mapping method, system, computer and readable storage medium | |
| CN107358645A (en) | Product method for reconstructing three-dimensional model and its system | |
| KR101602472B1 (en) | Apparatus and method for generating 3D printing file using 2D image converting | |
| CN112509106A (en) | Document picture flattening method, device and equipment | |
| CN113516719B (en) | Camera calibration method, system and storage medium based on multiple homography matrixes | |
| CN108696745A (en) | Camera calibrated | |
| CN111161348A (en) | Monocular camera-based object pose estimation method, device and equipment | |
| CN116612262B (en) | Automatic grid adjustment method, system, equipment and medium aligned with reference photo | |
| CN106934846B (en) | Cloth image processing method and system | |
| US20120038785A1 (en) | Method for producing high resolution image | |
| CN110458951B (en) | Modeling data acquisition method and related device for power grid pole tower | |
| CN101729739A (en) | Method for rectifying deviation of image | |
| JP6156922B2 (en) | Three-dimensional data generation apparatus, three-dimensional data generation method, and program | |
| CN115294277A (en) | Three-dimensional reconstruction method and device of object, electronic equipment and storage medium | |
| Bushnevskiy et al. | Multimode camera calibration | |
| CN117078839A (en) | Three-dimensional model reconstruction method, system and terminal based on nerve radiation field | |
| CN113593036A (en) | Imaging system, method, device and program with reference to grid parameterized model |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |