CN112330824A - Image processing method, image processing device, electronic equipment and storage medium - Google Patents
Image processing method, image processing device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN112330824A CN112330824A CN202011241111.3A CN202011241111A CN112330824A CN 112330824 A CN112330824 A CN 112330824A CN 202011241111 A CN202011241111 A CN 202011241111A CN 112330824 A CN112330824 A CN 112330824A
- Authority
- CN
- China
- Prior art keywords
- face
- dimensional
- human face
- image
- model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 34
- 238000012545 processing Methods 0.000 title claims abstract description 25
- 238000006073 displacement reaction Methods 0.000 claims abstract description 74
- 238000007493 shaping process Methods 0.000 claims abstract description 64
- 239000013598 vector Substances 0.000 claims abstract description 60
- 238000013507 mapping Methods 0.000 claims abstract description 23
- 238000004590 computer program Methods 0.000 claims description 6
- 238000004422 calculation algorithm Methods 0.000 claims description 4
- 238000004364 calculation method Methods 0.000 claims description 3
- 238000000034 method Methods 0.000 abstract description 17
- 238000010586 diagram Methods 0.000 description 10
- 230000006870 function Effects 0.000 description 5
- 238000012790 confirmation Methods 0.000 description 3
- 210000000216 zygoma Anatomy 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 230000004075 alteration Effects 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000013139 quantization Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/20—Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
- G06T17/20—Finite element generation, e.g. wire-frame surface description, tesselation
-
- G06T3/06—
-
- G06T3/18—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2219/00—Indexing scheme for manipulating 3D models or images for computer graphics
- G06T2219/20—Indexing scheme for editing of 3D models
- G06T2219/2021—Shape modification
Abstract
The application discloses an image processing method, an image processing device, electronic equipment and a storage medium. The method comprises the following steps: acquiring a preset shaping parameter on a target object human face three-dimensional grid model; calculating displacement vectors of all 3D key points in the human face three-dimensional grid model on a three-dimensional space based on shaping parameters on the human face three-dimensional grid model; for each key point, mapping the 3D displacement vector of each key point to a two-dimensional plane to obtain the displacement vector of each key point on the two-dimensional plane; and generating a 3D shaped face image of the target object according to the three-dimensional mesh model of the face and the displacement vector of each key point on the two-dimensional plane. The method can obtain a more accurate face shaping result, improves the accuracy of image deformation, and improves the use experience of users.
Description
Technical Field
The present application relates to the field of image processing technologies, and in particular, to an image processing method and apparatus, an electronic device, and a computer-readable storage medium.
Background
The existing face image reshaping technology is generally based on a 2D plane image, and pixel points on the plane image are shifted. In the related art, there are two main methods used for implementing face image reshaping: one is that a pixel displacement range is defined, a displacement weight is set for each pixel in the range, and then each pixel is displaced according to a certain direction; the other method is that the 2D plane image is divided into a plurality of triangles, the vertex of each triangle is displaced according to the required direction, and then the interior of the changed triangle is linearly stretched, so that the deformed image can be obtained.
But the problems that exist at present are: the two modes are that 2D key points of the human face are obtained from a 2D plane image, displacement variables are set according to the 2D key points, the displacement variables are relative values of the key points on the plane image, and the image deformation is not accurate enough due to low quantization degree.
Disclosure of Invention
The object of the present application is to solve at least to some extent one of the above mentioned technical problems.
To this end, a first object of the present application is to propose an image processing method. The method can obtain a more accurate face shaping result, improves the accuracy of image deformation, and improves the use experience of users.
A second object of the present application is to provide an image processing apparatus.
A third object of the present application is to provide an electronic device.
A fourth object of the present application is to propose a computer readable storage medium.
In order to achieve the above object, an embodiment of the first aspect of the present application provides an image processing method, including: acquiring a preset shaping parameter on a target object human face three-dimensional grid model; calculating displacement vectors of all 3D key points in the human face three-dimensional grid model on a three-dimensional space based on shaping parameters on the human face three-dimensional grid model; for each key point, mapping the 3D displacement vector of each key point to a two-dimensional plane to obtain the displacement vector of each key point on the two-dimensional plane; and generating a 3D shaped face image of the target object according to the face three-dimensional grid model and the displacement vector of each key point on the two-dimensional plane.
In order to achieve the above object, an embodiment of a second aspect of the present application provides an image processing apparatus, including: the shaping parameter acquisition module is used for acquiring a preset shaping parameter on the target object human face three-dimensional grid model; the calculation module is used for calculating displacement vectors of all 3D key points in the human face three-dimensional grid model on a three-dimensional space based on the shaping parameters on the human face three-dimensional grid model; the displacement vector mapping module is used for mapping the 3D displacement vector of each key point to a two-dimensional plane aiming at each key point to obtain the displacement vector of each key point on the two-dimensional plane; and the shaping image generation module is used for generating a 3D shaped face image of the target object according to the face three-dimensional grid model and the displacement vector of each key point on the two-dimensional plane.
In order to achieve the above object, an electronic device according to a third aspect of the present application includes: the image processing method comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein when the processor executes the program, the image processing method is realized.
To achieve the above object, a non-transitory computer-readable storage medium is provided in an embodiment of a fourth aspect of the present application, on which a computer program is stored, and the computer program, when executed by a processor, implements the image processing method according to the embodiment of the first aspect of the present application.
According to the image processing method, the image processing device, the electronic equipment and the storage medium, the 3D displacement vector of the 3D key point of the face can be obtained through the set shaping parameters on the three-dimensional mesh model of the face, the mapping relation between the 3D key point displacement vector and the 2D plane is calculated according to the orientation angle of the face on the image, the displacement vector on the 2D plane is obtained, the new face key point is calculated, and finally the image is linearly stretched according to the mesh structure of the 3D key point, so that the shaping result of the face can be obtained. The pixel displacement on the face image is quantized based on the face three-dimensional grid model, the displacement on the three-dimensional grid model is set at first, then the mapping from the 3D direction to the 2D direction is carried out, the pixel displacement on the quantized 2D image is obtained, and therefore a more accurate face shaping result is obtained, the accuracy of image deformation is improved, and the use experience of a user is improved.
Additional aspects and advantages of the present application will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the present application.
Drawings
The foregoing and/or additional aspects and advantages of the present application will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:
FIG. 1 is a flow diagram of an image processing method according to one embodiment of the present application;
FIG. 2 is a flow chart of a method for building a three-dimensional mesh model of a human face according to an embodiment of the application;
FIG. 3 is a flow diagram of an image processing method according to an embodiment of the present application;
FIG. 4 is a schematic diagram of an image processing apparatus according to an embodiment of the present application;
FIG. 5 is a schematic diagram of an image processing apparatus according to an embodiment of the present application;
FIG. 6 is a schematic diagram of an image processing apparatus according to another embodiment of the present application;
FIG. 7 is a schematic structural diagram of an electronic device according to one embodiment of the present application.
Detailed Description
Reference will now be made in detail to embodiments of the present application, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the drawings are exemplary and intended to be used for explaining the present application and should not be construed as limiting the present application.
An image processing method, an apparatus, an electronic device, and a computer-readable storage medium according to embodiments of the present application are described below with reference to the accompanying drawings.
FIG. 1 is a flow diagram of an image processing method according to one embodiment of the present application. It should be noted that the image processing method according to the embodiment of the present application is applicable to the image processing apparatus according to the embodiment of the present application, and the image processing apparatus may be configured on the electronic device according to the embodiment of the present application. The electronic device may be a mobile terminal, such as a mobile phone, a tablet computer, a personal digital assistant, and other hardware devices with various operating systems.
As shown in fig. 1, the image processing method may include:
and S110, acquiring a preset shaping parameter on the target object human face three-dimensional grid model.
Specifically, the shaping parameters on the three-dimensional mesh model of the face of the target object may be set in advance. As an example, the shaping parameters on the three-dimensional mesh model of the human face can be set by the following method: the method comprises the steps of obtaining a face three-dimensional grid model of a pre-established target object, receiving editing operation of the target object aiming at the face three-dimensional grid model, and determining shaping parameters on the face three-dimensional grid model according to the editing operation.
Optionally, before the face image of the target object is shaped, a three-dimensional mesh model of the face of the target object may be obtained. At this time, the target object may edit the three-dimensional mesh model of the human face according to the needs and preferences of the target object, for example, the three-dimensional mesh model of the human face may be operated to thin the face, bulge the nose, reduce the cheekbones, cushion the chin, and so on. When receiving the editing operation of the target object aiming at the face three-dimensional grid model, the shaping parameters on the face three-dimensional grid model can be determined according to the editing operation. The shaping parameters may include, but are not limited to, a pixel point to be processed and a displacement variable to be moved by the pixel point to be processed. For example, taking the editing operation as a face thinning operation as an example, it is determined which pixel points on the face three-dimensional mesh model need to be processed and to which the pixel points need to be moved according to the face thinning operation.
In one embodiment of the present application, as shown in fig. 2, a three-dimensional mesh model of a face of the target object may be established in advance by the following steps:
s210, acquiring a multi-frame image of a target object;
s220, constructing a human face three-dimensional model of the target object according to the multi-frame image;
s230, extracting a plurality of 3D key points on the face contour from the grid information based on the grid information of the face three-dimensional model;
for example, 296 3D key points on the face contour can be extracted from mesh information of a three-dimensional model of the face based on the mesh information.
S240, based on an interpolation algorithm, generating a plurality of 3D key points inside the human face and a plurality of 3D key points outside the human face according to the plurality of 3D key points on the human face contour;
and S250, establishing a human face three-dimensional mesh model of the target object according to the plurality of 3D key points on the human face contour, the plurality of 3D key points inside the human face and the plurality of 3D key points outside the human face contour.
For example, 33 face internal 3D key points and 8 face contour external 3D key points can be obtained by interpolation according to a plurality of 3D key points on the face contour, and a new 3D mesh model, that is, a face three-dimensional mesh model of the target object is constructed according to these 3D key points.
Therefore, the target object edits the human face three-dimensional grid model in advance, and the shaping parameters on the human face three-dimensional grid model can be determined according to the editing operation, so that the preset shaping parameters on the human face three-dimensional grid model can be obtained when the planar image of the target object is shaped, and then the planar image of the target object can be shaped subsequently based on the shaping parameters.
And S120, calculating displacement vectors of all 3D key points in the human face three-dimensional grid model in a three-dimensional space based on the shaping parameters on the human face three-dimensional grid model.
For example, taking the face-thinning operation performed on the face three-dimensional mesh model as an example, a shaping parameter corresponding to the face-thinning operation may be determined, for example, the shaping parameter may be information about which pixel points on the face three-dimensional mesh model need to be processed, to which the pixel points need to be moved, and the like, so that a displacement vector of each 3D key point in the face three-dimensional mesh model in a three-dimensional space may be calculated based on the shaping parameter.
And S130, aiming at each key point, mapping the 3D displacement vector of each key point to a two-dimensional plane to obtain the displacement vector of each key point on the two-dimensional plane.
Optionally, a face orientation angle on a two-dimensional plane is determined, a mapping relation from the 3D displacement vector of each key point to the two-dimensional plane is calculated according to the face orientation angle on the two-dimensional plane, and a displacement vector of each key point on the two-dimensional plane is determined according to the mapping relation.
That is to say, displacement vectors of each 3D keypoint in the three-dimensional space can be projected onto the two-dimensional plane according to the face orientation angle on the two-dimensional plane image, so as to obtain a projection vector of the 3D displacement vector of each keypoint on the two-dimensional plane, where the projection vector is a displacement vector of the keypoint on the two-dimensional plane.
And S140, generating a 3D shaped face image of the target object according to the three-dimensional mesh model of the face and the displacement vector of each key point on the two-dimensional plane.
As an example, the deformed 3D key points on the plane image may be generated according to the displacement vector of each key point on the two-dimensional plane, and the plane image is linearly stretched according to the three-dimensional mesh model of the face and the deformed 3D key points, so as to obtain a face image of the target object after 3D reshaping.
That is to say, the position of a key point to be deformed on a planar image can be calculated according to the displacement vector of each key point on the two-dimensional plane, and the planar image is linearly stretched according to the three-dimensional mesh model of the face and the position of the key point to be deformed of each key point, so as to obtain a face image of the target object after 3D reshaping.
According to the image processing method, the 3D displacement vector of the 3D key point of the face can be obtained through the set shaping parameters on the three-dimensional mesh model of the face, the mapping relation between the 3D key point displacement vector and the 2D plane is calculated according to the orientation angle of the face on the image, the displacement vector on the 2D plane is obtained, a new face key point is calculated, and finally the image is linearly stretched according to the mesh structure of the 3D key point, so that the shaping result of the face can be obtained. The pixel displacement on the face image is quantized based on the face three-dimensional grid model, the displacement on the three-dimensional grid model is set at first, then the mapping from the 3D direction to the 2D direction is carried out, the pixel displacement on the quantized 2D image is obtained, and therefore a more accurate face shaping result is obtained, the accuracy of image deformation is improved, and the use experience of a user is improved.
FIG. 3 is a flow diagram of an image processing method according to an embodiment of the present application.
In order to improve user experience, the image processing method of the embodiment of the application can be applied to a self-photographing scene of a user, for example, when the user is detected to use a self-photographing mode, the image displayed on a preview interface can be automatically reshaped, and thus when the user is detected to click a shooting confirmation button, the reshaped face image can be directly obtained. Specifically, as shown in fig. 3, the image processing method may include:
s310, when the user is detected to use the self-photographing mode, the preset shaping parameters on the user face three-dimensional grid model are obtained.
Specifically, shaping parameters on the three-dimensional mesh model of the face of the user can be preset. As an example, the shaping parameters on the three-dimensional mesh model of the human face can be set by the following method: the method comprises the steps of obtaining a pre-established face three-dimensional grid model of a user, receiving editing operation of the user aiming at the face three-dimensional grid model, and determining shaping parameters on the face three-dimensional grid model according to the editing operation.
Optionally, before the face image of the user is shaped, a three-dimensional mesh model of the face of the user may be obtained. At this time, the user may edit the three-dimensional mesh model of the face according to the needs and preferences of the user, for example, the user may perform operations such as face thinning, nose humping, cheekbone reduction, chin cushioning, and the like on the three-dimensional mesh model of the face. When receiving the editing operation of the user aiming at the face three-dimensional grid model, the shaping parameters on the face three-dimensional grid model can be determined according to the editing operation. The shaping parameters may include, but are not limited to, a pixel point to be processed and a displacement variable to be moved by the pixel point to be processed. For example, taking the editing operation as a face thinning operation as an example, it is determined which pixel points on the face three-dimensional mesh model need to be processed and to which the pixel points need to be moved according to the face thinning operation.
And S320, calculating displacement vectors of all 3D key points in the human face three-dimensional grid model on a three-dimensional space based on the shaping parameters on the human face three-dimensional grid model.
S330, aiming at each key point, mapping the 3D displacement vector of each key point to a two-dimensional plane to obtain the displacement vector of each key point on the two-dimensional plane.
And S340, generating a 3D shaped face image of the user according to the three-dimensional face grid model and the displacement vector of each key point on the two-dimensional plane.
As an example, the deformed 3D key points on the plane image in the shooting preview interface may be generated according to the displacement vector of each key point on the two-dimensional plane, and the plane image is linearly stretched according to the three-dimensional mesh model of the face and the deformed 3D key points, so as to obtain a face image of the user after 3D reshaping.
And S350, displaying the face image of the user after 3D reshaping on a shooting preview interface.
Optionally, after the face image after 3D reshaping by the user is obtained, the face image may be displayed on a shooting preview interface, so that when it is detected that the user clicks a shooting confirmation button, the reshaped face image may be directly obtained.
According to the image processing method, when the user is detected to use the self-photographing mode, the image displayed on the preview interface can be automatically shaped, so that when the user is detected to click the shooting confirmation button, the shaped face image can be directly obtained, and the photographing experience of the user is improved.
In an embodiment of the present application, the image processing method according to the embodiment of the present application may also be applied to a scene in which a planar image obtained by shooting is subjected to a shaping process, for example, the image processing method according to the embodiment of the present application may be used to perform a shaping process on a self-timer image that has been shot by a user, so as to obtain a 3D shaped self-timer image.
In accordance with the image processing methods provided in the foregoing embodiments, an embodiment of the present application further provides an image processing apparatus, and since the image processing apparatus provided in the embodiment of the present application corresponds to the image processing methods provided in the foregoing embodiments, the embodiments of the image processing method described above are also applicable to the image processing apparatus provided in the embodiment, and are not described in detail in the embodiment. Fig. 4 is a schematic structural diagram of an image processing apparatus according to an embodiment of the present application. As shown in fig. 4, the image processing apparatus 400 may include: a shaping parameter obtaining module 410, a calculating module 420, a displacement vector mapping module 430 and a shaped image generating module 440.
Specifically, the shaping parameter obtaining module 410 is configured to obtain a preset shaping parameter on the target object human face three-dimensional mesh model.
The calculation module 420 is configured to calculate a displacement vector of each 3D key point in the three-dimensional face mesh model in the three-dimensional space based on the shaping parameter on the three-dimensional face mesh model.
The displacement vector mapping module 430 is configured to map, for each keypoint, the 3D displacement vector of each keypoint onto a two-dimensional plane, so as to obtain a displacement vector of each keypoint on the two-dimensional plane. As an example, the displacement vector mapping module 430 may determine a face orientation angle on the two-dimensional plane, calculate a mapping relationship of the 3D displacement vector of each keypoint to the two-dimensional plane according to the face orientation angle on the two-dimensional plane, and determine a displacement vector of each keypoint on the two-dimensional plane according to the mapping relationship.
The reshaped image generation module 440 is configured to generate a 3D reshaped face image of the target object according to the three-dimensional mesh model of the face and the displacement vector of each key point on the two-dimensional plane. As an example, the reshaped image generating module 440 may generate a deformed 3D key point on a plane image according to a displacement vector of each key point on the two-dimensional plane, and linearly stretch the plane image according to the three-dimensional mesh model of the human face and the deformed 3D key point, so as to obtain a human face image of the target object after 3D reshaping.
It should be noted that, the shaping parameters on the three-dimensional mesh model of the face of the target object may be preset. Optionally, in an embodiment of the present application, as shown in fig. 5, the image processing apparatus 400 may further include: a shaping parameter presetting module 450, configured to preset shaping parameters on the target object face three-dimensional mesh model. In an embodiment of the present application, as shown in fig. 5, the shaping parameter presetting module 450 may include: an acquisition unit 451, a reception unit 452, and a determination unit 453. The obtaining unit 451 is used for obtaining a pre-established human face three-dimensional mesh model of the target object; the receiving unit 452 is configured to receive an editing operation of the target object on the human face three-dimensional mesh model; the determining unit 453 is configured to determine a shaping parameter on the face three-dimensional mesh model according to the editing operation.
It should be further noted that, before the face image of the target object is reshaped, a three-dimensional mesh model of the face of the target object may be obtained. At this time, the target object may edit the three-dimensional mesh model of the human face according to the needs and preferences of the target object, for example, the three-dimensional mesh model of the human face may be operated to thin the face, bulge the nose, reduce the cheekbones, cushion the chin, and so on. When receiving the editing operation of the target object aiming at the face three-dimensional grid model, the shaping parameters on the face three-dimensional grid model can be determined according to the editing operation. The shaping parameters may include, but are not limited to, a pixel point to be processed and a displacement variable to be moved by the pixel point to be processed. For example, taking the editing operation as a face thinning operation as an example, it is determined which pixel points on the face three-dimensional mesh model need to be processed and to which the pixel points need to be moved according to the face thinning operation.
Optionally, in an embodiment of the present application, as shown in fig. 6, the image processing apparatus 400 may further include: a model pre-establishing module 460, configured to pre-establish a three-dimensional mesh model of the face of the target object. In an embodiment of the present application, as shown in fig. 6, the model pre-building module 460 may include: an acquisition unit 461, a first establishing unit 462, an extraction unit 463, a generation unit 464, and a second establishing unit 465.
Wherein the obtaining unit 461 is configured to obtain a multi-frame image of the target object; the first establishing unit 462 is configured to establish a three-dimensional model of a human face of the target object according to the multi-frame images; the extracting unit 463 is configured to extract a plurality of 3D key points on a face contour from mesh information of the three-dimensional face model based on the mesh information; the generating unit 464 is configured to generate a plurality of 3D key points inside the face and a plurality of 3D key points outside the face contour according to the plurality of 3D key points on the face contour based on an interpolation algorithm; the second establishing unit 465 is configured to establish a three-dimensional mesh model of the face of the target object according to the plurality of 3D key points on the face contour, the plurality of 3D key points inside the face, and the plurality of 3D key points outside the face contour.
According to the image processing device of the embodiment of the application, the displacement of the pixel points on the face image can be quantized based on the face three-dimensional grid model, the displacement on the three-dimensional grid model is firstly set, then the mapping from the 3D to the 2D direction is carried out, and the pixel displacement on the quantized 2D image is obtained, so that a more accurate face shaping result is obtained, the accuracy of image deformation is improved, and the use experience of a user is improved.
In order to implement the above embodiments, the present application further provides an electronic device.
FIG. 7 is a schematic structural diagram of an electronic device according to one embodiment of the present application. As shown in fig. 7, the electronic device 700 may include: the image processing method according to any of the above embodiments of the present application is implemented when the processor 720 executes the program 730, and the processor 720 executes the program 730.
In order to achieve the above embodiments, the present application also proposes a non-transitory computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the image processing method according to any of the above embodiments of the present application.
In the description of the present application, it is to be understood that the terms "first", "second" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implying any number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present application, "plurality" means at least two, e.g., two, three, etc., unless specifically limited otherwise.
In the description herein, reference to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the application. In this specification, the schematic representations of the terms used above are not necessarily intended to refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, various embodiments or examples and features of different embodiments or examples described in this specification can be combined and combined by one skilled in the art without contradiction.
Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or steps of the process, and the scope of the preferred embodiments of the present application includes other implementations in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the present application.
The logic and/or steps represented in the flowcharts or otherwise described herein, e.g., an ordered listing of executable instructions that can be considered to implement logical functions, can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. For the purposes of this description, a "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer-readable medium would include the following: an electrical connection (electronic device) having one or more wires, a portable computer diskette (magnetic device), a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium could even be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, via for instance optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner if necessary, and then stored in a computer memory.
It should be understood that portions of the present application may be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one or combination of the following techniques, which are known in the art, may be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application specific integrated circuit having an appropriate combinational logic gate circuit, a Programmable Gate Array (PGA), a Field Programmable Gate Array (FPGA), or the like.
It will be understood by those skilled in the art that all or part of the steps carried by the method for implementing the above embodiments may be implemented by hardware related to instructions of a program, which may be stored in a computer readable storage medium, and when the program is executed, the program includes one or a combination of the steps of the method embodiments.
In addition, functional units in the embodiments of the present application may be integrated into one processing module, or each unit may exist alone physically, or two or more units are integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode. The integrated module, if implemented in the form of a software functional module and sold or used as a stand-alone product, may also be stored in a computer readable storage medium.
The storage medium mentioned above may be a read-only memory, a magnetic or optical disk, etc. Although embodiments of the present application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present application, and that variations, modifications, substitutions and alterations may be made to the above embodiments by those of ordinary skill in the art within the scope of the present application.
Claims (10)
1. An image processing method, characterized by comprising the steps of:
responding to the operation aiming at the human face three-dimensional grid model, and acquiring shaping parameters on the human face three-dimensional grid model;
calculating displacement vectors of all 3D key points in the human face three-dimensional grid model on a three-dimensional space based on shaping parameters on the human face three-dimensional grid model;
determining a face orientation angle on a two-dimensional plane image;
aiming at each key point, calculating the mapping relation between the 3D displacement vector of each key point and a two-dimensional plane according to the face orientation angle on the two-dimensional plane image;
determining a displacement vector of each key point on the two-dimensional plane according to the mapping relation;
and generating a 3D shaped face image according to the three-dimensional mesh model of the face and the displacement vector of each key point on the two-dimensional plane.
2. The image processing method of claim 1, wherein the shaping parameters on the face three-dimensional mesh model are preset by:
acquiring a pre-established human face three-dimensional grid model;
receiving an editing operation aiming at the human face three-dimensional grid model;
and determining a shaping parameter on the human face three-dimensional grid model according to the editing operation.
3. The image processing method of claim 2, wherein the face three-dimensional mesh model is built by:
acquiring a multi-frame image;
constructing a human face three-dimensional model according to the multi-frame images;
extracting a plurality of 3D key points on the face contour from the grid information based on the grid information of the face three-dimensional model;
based on an interpolation algorithm, generating a plurality of 3D key points inside the human face and a plurality of 3D key points outside the human face according to the plurality of 3D key points on the human face contour;
and establishing the human face three-dimensional mesh model according to the plurality of 3D key points on the human face contour, the plurality of 3D key points inside the human face and the plurality of 3D key points outside the human face contour.
4. The image processing method of claim 1, wherein the generating of the 3D reshaped face image according to the three-dimensional mesh model of the face and the displacement vector of each key point on the two-dimensional plane comprises:
generating deformed 3D key points on the plane image according to the displacement vector of each key point on the two-dimensional plane;
and linearly stretching the plane image according to the three-dimensional mesh model of the face and the deformed 3D key point to obtain a face image subjected to 3D shaping.
5. An image processing apparatus characterized by comprising:
the shaping parameter acquisition module is used for responding to the operation aiming at the human face three-dimensional grid model and acquiring shaping parameters on the human face three-dimensional grid model;
the calculation module is used for calculating displacement vectors of all 3D key points in the human face three-dimensional grid model on a three-dimensional space based on the shaping parameters on the human face three-dimensional grid model;
the displacement vector mapping module is used for determining a face orientation angle on a two-dimensional plane image, calculating a mapping relation from a 3D displacement vector of each key point to a two-dimensional plane according to the face orientation angle on the two-dimensional plane image aiming at each key point, and determining a displacement vector of each key point on the two-dimensional plane according to the mapping relation;
and the shaping image generation module is used for generating a 3D shaped face image according to the face three-dimensional grid model and the displacement vector of each key point on the two-dimensional plane.
6. The image processing apparatus according to claim 5, wherein said apparatus further comprises:
the shaping parameter presetting module is used for presetting shaping parameters on the human face three-dimensional grid model;
wherein, the shaping parameter presetting module comprises:
the acquisition unit is used for acquiring the pre-established human face three-dimensional grid model;
the receiving unit is used for receiving editing operation aiming at the human face three-dimensional grid model;
and the determining unit is used for determining the shaping parameters on the human face three-dimensional mesh model according to the editing operation.
7. The image processing apparatus according to claim 6, wherein said apparatus further comprises:
the model pre-establishing module is used for pre-establishing the human face three-dimensional grid model;
wherein the model pre-establishing module comprises:
an acquisition unit configured to acquire a plurality of frame images;
the first establishing unit is used for establishing a human face three-dimensional model according to the multi-frame images;
the extracting unit is used for extracting a plurality of 3D key points on the face contour from the grid information based on the grid information of the face three-dimensional model;
the generating unit is used for generating a plurality of 3D key points inside the human face and a plurality of 3D key points outside the human face according to the plurality of 3D key points on the human face contour based on an interpolation algorithm;
and the second establishing unit is used for establishing the human face three-dimensional mesh model according to the plurality of 3D key points on the human face contour, the plurality of 3D key points inside the human face and the plurality of 3D key points outside the human face contour.
8. The image processing apparatus of claim 5, wherein the reshaped image generation module is specifically configured to:
generating deformed 3D key points on the plane image according to the displacement vector of each key point on the two-dimensional plane;
and linearly stretching the plane image according to the three-dimensional mesh model of the face and the deformed 3D key point to obtain a face image subjected to 3D shaping.
9. An electronic device, comprising: memory, processor and computer program stored on the memory and executable on the processor, which when executed by the processor implements the image processing method according to any of claims 1 to 4.
10. A non-transitory computer-readable storage medium on which a computer program is stored, the program, when executed by a processor, implementing the image processing method according to any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011241111.3A CN112330824A (en) | 2018-05-31 | 2018-05-31 | Image processing method, image processing device, electronic equipment and storage medium |
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011241111.3A CN112330824A (en) | 2018-05-31 | 2018-05-31 | Image processing method, image processing device, electronic equipment and storage medium |
| CN201810551066.8A CN108765351B (en) | 2018-05-31 | 2018-05-31 | Image processing method, image processing device, electronic equipment and storage medium |
Related Parent Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810551066.8A Division CN108765351B (en) | 2018-05-31 | 2018-05-31 | Image processing method, image processing device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112330824A true CN112330824A (en) | 2021-02-05 |
Family
ID=64001235
Family Applications (2)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810551066.8A Active CN108765351B (en) | 2018-05-31 | 2018-05-31 | Image processing method, image processing device, electronic equipment and storage medium |
| CN202011241111.3A Pending CN112330824A (en) | 2018-05-31 | 2018-05-31 | Image processing method, image processing device, electronic equipment and storage medium |
Family Applications Before (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810551066.8A Active CN108765351B (en) | 2018-05-31 | 2018-05-31 | Image processing method, image processing device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (2) | CN108765351B (en) |
Families Citing this family (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111445568B (en) * | 2018-12-28 | 2023-08-15 | 广州市百果园网络科技有限公司 | Character expression editing method, device, computer storage medium and terminal |
| CN109741277A (en) * | 2018-12-29 | 2019-05-10 | 广州华多网络科技有限公司 | Image processing method, device, storage medium and server |
| CN109711390A (en) * | 2019-01-17 | 2019-05-03 | 深圳英飞拓科技股份有限公司 | Face scratches the preferred method and device of figure picture |
| CN110060348B (en) * | 2019-04-26 | 2023-08-11 | 北京迈格威科技有限公司 | Face image shaping method and device |
| CN110060287B (en) * | 2019-04-26 | 2021-06-15 | 北京迈格威科技有限公司 | Face image nose shaping method and device |
| CN110457846B (en) * | 2019-08-16 | 2023-08-18 | 广东商鼎智能设备有限公司 | Electric spark machining parameter generation method and device, electronic equipment and storage medium |
| CN110675489B (en) * | 2019-09-25 | 2024-01-23 | 北京达佳互联信息技术有限公司 | Image processing method, device, electronic equipment and storage medium |
| CN111031305A (en) * | 2019-11-21 | 2020-04-17 | 北京市商汤科技开发有限公司 | Image processing method and apparatus, image device, and storage medium |
| JP2022512262A (en) | 2019-11-21 | 2022-02-03 | ベイジン センスタイム テクノロジー デベロップメント カンパニー， リミテッド | Image processing methods and equipment, image processing equipment and storage media |
| CN113570634A (en) * | 2020-04-28 | 2021-10-29 | 北京达佳互联信息技术有限公司 | Object three-dimensional reconstruction method and device, electronic equipment and storage medium |
| CN111467131B (en) * | 2020-04-30 | 2022-05-27 | 浙江大学 | Automatic design method for 3D printing customized goggles frame |
| CN111754431B (en) * | 2020-06-17 | 2023-08-01 | 北京百度网讯科技有限公司 | Image area replacement method, device, equipment and storage medium |
| CN112241933A (en) * | 2020-07-15 | 2021-01-19 | 北京沃东天骏信息技术有限公司 | Face image processing method and device, storage medium and electronic equipment |
| JP7214007B2 (en) | 2020-09-22 | 2023-01-27 | 深▲セン▼市衡泰信科技有限公司 | GOLF BALL ON-TOP DETECTION METHOD, SYSTEM AND STORAGE MEDIUM |
| CN112614228B (en) * | 2020-12-17 | 2023-09-05 | 北京达佳互联信息技术有限公司 | Method, device, electronic equipment and storage medium for simplifying three-dimensional grid |
| CN112669447B (en) * | 2020-12-30 | 2023-06-30 | 网易（杭州）网络有限公司 | Model head portrait creation method and device, electronic equipment and storage medium |
| CN113362231A (en) * | 2021-07-23 | 2021-09-07 | 百果园技术(新加坡)有限公司 | Interpolation method and device for key points of human face, computer equipment and storage medium |
| CN115409951B (en) * | 2022-10-28 | 2023-03-24 | 北京百度网讯科技有限公司 | Image processing method, image processing device, electronic equipment and storage medium |
Citations (22)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101101672A (en) * | 2007-07-13 | 2008-01-09 | 中国科学技术大学 | Stereo vision three-dimensional human face modelling approach based on dummy image |
| CN101311966A (en) * | 2008-06-20 | 2008-11-26 | 浙江大学 | Three-dimensional human face animations editing and synthesis a based on operation transmission and Isomap analysis |
| CN101320484A (en) * | 2008-07-17 | 2008-12-10 | 清华大学 | Three-dimensional human face recognition method based on human face full-automatic positioning |
| CN101383055A (en) * | 2008-09-18 | 2009-03-11 | 北京中星微电子有限公司 | Three-dimensional human face constructing method and system |
| CN101561874A (en) * | 2008-07-17 | 2009-10-21 | 清华大学 | Method for recognizing face images |
| CN101739719A (en) * | 2009-12-24 | 2010-06-16 | 四川大学 | Three-dimensional gridding method of two-dimensional front view human face image |
| CN101777195A (en) * | 2010-01-29 | 2010-07-14 | 浙江大学 | Three-dimensional face model adjusting method |
| CN104036546A (en) * | 2014-06-30 | 2014-09-10 | 清华大学 | Method for carrying out face three-dimensional reconstruction at any viewing angle on basis of self-adaptive deformable model |
| CN105787878A (en) * | 2016-02-25 | 2016-07-20 | 杭州格像科技有限公司 | Beauty processing method and device |
| CN105913416A (en) * | 2016-04-06 | 2016-08-31 | 中南大学 | Method for automatically segmenting three-dimensional human face model area |
| CN105938627A (en) * | 2016-04-12 | 2016-09-14 | 湖南拓视觉信息技术有限公司 | Processing method and system for virtual plastic processing on face |
| CN106503671A (en) * | 2016-11-03 | 2017-03-15 | 厦门中控生物识别信息技术有限公司 | The method and apparatus for determining human face posture |
| CN106920274A (en) * | 2017-01-20 | 2017-07-04 | 南京开为网络科技有限公司 | Mobile terminal 2D key points rapid translating is the human face model building of 3D fusion deformations |
| CN107464212A (en) * | 2017-07-28 | 2017-12-12 | 广东欧珀移动通信有限公司 | U.S. face method, electronic installation and computer-readable recording medium |
| CN107730465A (en) * | 2017-10-09 | 2018-02-23 | 武汉斗鱼网络科技有限公司 | Face U.S. face method and device in a kind of image |
| CN107730445A (en) * | 2017-10-31 | 2018-02-23 | 广东欧珀移动通信有限公司 | Image processing method, device, storage medium and electronic equipment |
| CN107886484A (en) * | 2017-11-30 | 2018-04-06 | 广东欧珀移动通信有限公司 | U.S. face method, apparatus, computer-readable recording medium and electronic equipment |
| US20180108183A1 (en) * | 2016-10-18 | 2018-04-19 | Microsoft Technology Licensing, Llc | Projecting a two-dimensional image onto a three-dimensional graphical object |
| CN107944435A (en) * | 2017-12-27 | 2018-04-20 | 广州图语信息科技有限公司 | A kind of three-dimensional face identification method, device and processing terminal |
| CN107948499A (en) * | 2017-10-31 | 2018-04-20 | 维沃移动通信有限公司 | A kind of image capturing method and mobile terminal |
| WO2018076437A1 (en) * | 2016-10-25 | 2018-05-03 | 宇龙计算机通信科技(深圳)有限公司 | Method and apparatus for human facial mapping |
| US20190228556A1 (en) * | 2016-09-21 | 2019-07-25 | Intel Corporation | Estimating accurate face shape and texture from an image |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130173235A1 (en) * | 2010-05-21 | 2013-07-04 | My Orthodontics Pty Ltd | Prediction of post-procedural appearance |
| CN103077553A (en) * | 2012-12-28 | 2013-05-01 | 海纳医信（北京）软件科技有限责任公司 | Three-dimensional coordinate determination method and device |
| CN106909875B (en) * | 2016-09-12 | 2020-04-10 | 湖南拓视觉信息技术有限公司 | Face type classification method and system |
-
2018
- 2018-05-31 CN CN201810551066.8A patent/CN108765351B/en active Active
- 2018-05-31 CN CN202011241111.3A patent/CN112330824A/en active Pending
Patent Citations (22)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101101672A (en) * | 2007-07-13 | 2008-01-09 | 中国科学技术大学 | Stereo vision three-dimensional human face modelling approach based on dummy image |
| CN101311966A (en) * | 2008-06-20 | 2008-11-26 | 浙江大学 | Three-dimensional human face animations editing and synthesis a based on operation transmission and Isomap analysis |
| CN101320484A (en) * | 2008-07-17 | 2008-12-10 | 清华大学 | Three-dimensional human face recognition method based on human face full-automatic positioning |
| CN101561874A (en) * | 2008-07-17 | 2009-10-21 | 清华大学 | Method for recognizing face images |
| CN101383055A (en) * | 2008-09-18 | 2009-03-11 | 北京中星微电子有限公司 | Three-dimensional human face constructing method and system |
| CN101739719A (en) * | 2009-12-24 | 2010-06-16 | 四川大学 | Three-dimensional gridding method of two-dimensional front view human face image |
| CN101777195A (en) * | 2010-01-29 | 2010-07-14 | 浙江大学 | Three-dimensional face model adjusting method |
| CN104036546A (en) * | 2014-06-30 | 2014-09-10 | 清华大学 | Method for carrying out face three-dimensional reconstruction at any viewing angle on basis of self-adaptive deformable model |
| CN105787878A (en) * | 2016-02-25 | 2016-07-20 | 杭州格像科技有限公司 | Beauty processing method and device |
| CN105913416A (en) * | 2016-04-06 | 2016-08-31 | 中南大学 | Method for automatically segmenting three-dimensional human face model area |
| CN105938627A (en) * | 2016-04-12 | 2016-09-14 | 湖南拓视觉信息技术有限公司 | Processing method and system for virtual plastic processing on face |
| US20190228556A1 (en) * | 2016-09-21 | 2019-07-25 | Intel Corporation | Estimating accurate face shape and texture from an image |
| US20180108183A1 (en) * | 2016-10-18 | 2018-04-19 | Microsoft Technology Licensing, Llc | Projecting a two-dimensional image onto a three-dimensional graphical object |
| WO2018076437A1 (en) * | 2016-10-25 | 2018-05-03 | 宇龙计算机通信科技(深圳)有限公司 | Method and apparatus for human facial mapping |
| CN106503671A (en) * | 2016-11-03 | 2017-03-15 | 厦门中控生物识别信息技术有限公司 | The method and apparatus for determining human face posture |
| CN106920274A (en) * | 2017-01-20 | 2017-07-04 | 南京开为网络科技有限公司 | Mobile terminal 2D key points rapid translating is the human face model building of 3D fusion deformations |
| CN107464212A (en) * | 2017-07-28 | 2017-12-12 | 广东欧珀移动通信有限公司 | U.S. face method, electronic installation and computer-readable recording medium |
| CN107730465A (en) * | 2017-10-09 | 2018-02-23 | 武汉斗鱼网络科技有限公司 | Face U.S. face method and device in a kind of image |
| CN107730445A (en) * | 2017-10-31 | 2018-02-23 | 广东欧珀移动通信有限公司 | Image processing method, device, storage medium and electronic equipment |
| CN107948499A (en) * | 2017-10-31 | 2018-04-20 | 维沃移动通信有限公司 | A kind of image capturing method and mobile terminal |
| CN107886484A (en) * | 2017-11-30 | 2018-04-06 | 广东欧珀移动通信有限公司 | U.S. face method, apparatus, computer-readable recording medium and electronic equipment |
| CN107944435A (en) * | 2017-12-27 | 2018-04-20 | 广州图语信息科技有限公司 | A kind of three-dimensional face identification method, device and processing terminal |
Non-Patent Citations (3)
| Title |
|---|
| 丁宾;孙天凯;鞠训光;王阳生;: "基于单张图像的三维人脸建模与表情动画", 计算机工程与设计, no. 07, 16 July 2012 (2012-07-16) * |
| 王琨等: "基于SFM算法的三维人脸模型重建", 计算机学报, vol. 28, no. 6, 30 June 2005 (2005-06-30) * |
| 高昊烨;汤艺璇;陈江源;叶超;肖南峰;: "基于2维正面人脸图像的3维人脸重构设计与实现", 重庆理工大学学报(自然科学), no. 04, 15 April 2018 (2018-04-15) * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108765351B (en) | 2020-12-08 |
| CN108765351A (en) | 2018-11-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108765351B (en) | Image processing method, image processing device, electronic equipment and storage medium | |
| CN110070611B (en) | Face three-dimensional reconstruction method and device based on depth image fusion | |
| JP6351238B2 (en) | Image processing apparatus, imaging apparatus, and distance correction method | |
| CN107705333B (en) | Space positioning method and device based on binocular camera | |
| KR101560508B1 (en) | Method and arrangement for 3-dimensional image model adaptation | |
| US8817075B2 (en) | Imaging apparatus capable of generating three-dimensional images, three-dimensional image generating method, and recording medium | |
| US9183634B2 (en) | Image processing apparatus and image processing method | |
| CN110971827B (en) | Portrait mode shooting method and device, terminal equipment and storage medium | |
| CN111105366B (en) | Image processing method and device, terminal equipment and storage medium | |
| CN115205450A (en) | Three-dimensional scanning data processing method, device, system, equipment and medium | |
| CN113033442B (en) | StyleGAN-based high-freedom face driving method and device | |
| EP4135317A2 (en) | Stereoscopic image acquisition method, electronic device and storage medium | |
| CN111105370A (en) | Image processing method, image processing apparatus, electronic device, and readable storage medium | |
| KR102389916B1 (en) | Method, apparatus, and device for identifying human body and computer readable storage | |
| WO2021192782A1 (en) | Dimension measurement method and dimension measurement device | |
| CN107534736B (en) | Image registration method and device of terminal and terminal | |
| CN113674139A (en) | Face image processing method and device, electronic equipment and storage medium | |
| CN109328459B (en) | Intelligent terminal, 3D imaging method thereof and 3D imaging system | |
| JP2002077941A (en) | Apparatus and method for generating depth image as well as computer readable recording medium recording program to execute the method in computer | |
| JP6704700B2 (en) | Image processing apparatus, control method thereof, and program | |
| JP3149923B2 (en) | Curve length measuring device, measuring method thereof, and storage medium storing control program for measuring curve length | |
| CN115471399B (en) | Panoramic image generation method, device, equipment and storage medium | |
| CN112529848B (en) | Panorama updating method and device, readable medium and equipment | |
| CN111182198A (en) | Shooting focusing method based on double cameras, mobile device and device | |
| CN112785533B (en) | Image fusion method, image fusion device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |