CN111583280B - Image processing method, device, equipment and computer readable storage medium - Google Patents
Image processing method, device, equipment and computer readable storage medium Download PDFInfo
- Publication number
- CN111583280B CN111583280B CN202010402093.6A CN202010402093A CN111583280B CN 111583280 B CN111583280 B CN 111583280B CN 202010402093 A CN202010402093 A CN 202010402093A CN 111583280 B CN111583280 B CN 111583280B
- Authority
- CN
- China
- Prior art keywords
- image
- face image
- face
- target
- average
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 30
- 238000012545 processing Methods 0.000 claims abstract description 64
- 230000009466 transformation Effects 0.000 claims abstract description 63
- 230000000694 effects Effects 0.000 claims abstract description 60
- 239000011159 matrix material Substances 0.000 claims abstract description 60
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 claims abstract description 50
- 238000005520 cutting process Methods 0.000 claims abstract description 40
- 238000000034 method Methods 0.000 claims abstract description 31
- 238000004364 calculation method Methods 0.000 claims description 6
- 210000004209 hair Anatomy 0.000 claims description 5
- 230000001815 facial effect Effects 0.000 abstract description 7
- 238000010586 diagram Methods 0.000 description 13
- 238000013461 design Methods 0.000 description 7
- 230000008569 process Effects 0.000 description 5
- 238000004891 communication Methods 0.000 description 4
- 238000012935 Averaging Methods 0.000 description 3
- 210000000887 face Anatomy 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 210000004709 eyebrow Anatomy 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000011022 operating instruction Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G06T3/04—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20112—Image segmentation details
- G06T2207/20132—Image cropping
Abstract
The invention provides an image processing method, an image processing device, image processing equipment and a computer readable storage medium, wherein the method comprises the following steps: acquiring an image cutting instruction sent by terminal equipment, wherein the image cutting instruction comprises a face image to be cut; determining a target transformation affine matrix according to at least one key point corresponding to a preset average face image and the face image; cutting the face image according to the target transformation affine matrix to obtain a cut target image; and sending the target image to the terminal equipment so that the user can edit the target image. The average face is generated according to a large number of face images, so that the facial features of most people can be represented, the face images are cut according to the target transformation affine matrix determined by the average face, the face cutting precision can be effectively improved, the effect of special effect processing according to the cut target images is better, the image processing requirements of users can be better met, and the user experience is improved.
Description
Technical Field
The present invention relates to the field of image processing, and in particular, to an image processing method, an image processing apparatus, an image processing device, and a computer-readable storage medium.
Background
With the development of science and technology, more and more application software goes into the life of users, and the amateur life of the users is gradually enriched, such as short video APP and the like. The user can record life by adopting modes of videos, photos and the like and upload the life on the short-time video APP.
In order to improve user experience, the existing short video APP generally performs special effect processing on a video uploaded by a user, for example, a sticker may be added on a face of the user. Before the special effect processing is carried out on the face, an accurate face region needs to be obtained, and the existing face region cutting method is often low in cutting accuracy, so that the content of a target region is lost, or too much content of a non-target region is reserved, and the effect of subsequent special effect processing is poor.
Disclosure of Invention
The invention provides an image processing method, an image processing device, image processing equipment and a computer readable storage medium, which are used for solving the technical problem that the subsequent special effect processing effect is poor due to low face region cutting precision in the existing image processing method.
A first aspect of the present invention provides an image processing method, including:
acquiring an image cutting instruction sent by terminal equipment, wherein the image cutting instruction comprises a face image to be cut;
determining a target transformation affine matrix according to at least one key point corresponding to a preset average face image and the face image;
cutting the face image according to the target transformation affine matrix to obtain a cut target image;
and sending the target image to the terminal equipment so as to enable a user to edit the target image.
According to the image processing method provided by the embodiment, a target transformation affine matrix is determined according to at least one average key point corresponding to a preset average face according to an image clipping instruction sent by the terminal device, a clipping operation is performed on a face image to be clipped according to the target transformation affine matrix, and the clipped target image is sent to the terminal device to be displayed. The average face is generated according to a large number of face images, so that the facial features of most people can be represented, the face images are cut according to the target transformation affine matrix determined by the average face, the face cutting precision can be effectively improved, the effect of special effect processing according to the cut target images is better, the image processing requirements of users can be better met, and the user experience is improved.
In one possible design, the determining a target transformation affine matrix according to at least one key point corresponding to a preset average face image and the face image includes:
determining a target transformation affine matrix according to a preset transformation formula, at least one key point corresponding to a preset average face image and the face image, wherein the transformation formula is as follows:
u＝a1x+b1y+c1
v＝a2x+b2y+c2
wherein, (x, y) is the coordinate of any one point in the target key points in the face image, and (u, v) is the average coordinate of the key points corresponding to the target key points in the average face image.
In a possible design, before determining a target transformation affine matrix according to at least one key point corresponding to a preset average face image and the face image, the method further includes:
acquiring a preset first face set, wherein first face images in the first face set are consistent in size and all comprise at least one key point;
determining at least one key point coordinate corresponding to each first face image in the first face combination;
calculating the average coordinate of each key point corresponding to the first face set according to at least one key point coordinate corresponding to each first face image;
performing deformation operation on each first face image according to the average coordinate of each key point to obtain a deformed first face image, wherein the distance between each key point coordinate in the deformed first face image and the average coordinate is smaller than a preset threshold value;
and determining an average face image corresponding to the deformed first face image set, wherein the pixel value of each pixel point in the average face image is the average value of the pixel values of corresponding pixel points in each deformed first face image in the deformed first face image set.
In the image processing method provided by this embodiment, the average coordinates of the key points indicating the same object are determined by the coordinates of the extracted key points for each first face image in the first face image set, and on the basis of the obtained average coordinates, the first face image is subjected to image deformation to obtain the deformed first face image set, thereby generating the average face image of the deformed first face image set. In the process of generating the average face image, triangulation on each first face image is not needed, so that the calculation amount of a server is reduced, and a basis is provided for the generation of a subsequent target affine transformation matrix.
In one possible design, the obtaining a preset first set of faces includes:
acquiring a preset second face image set;
and cutting the second face image with the key points in the second face image set according to a preset target size to obtain the first face image set.
In one possible design, the warping operation on each first face image according to the average coordinates of the key points includes:
and performing deformation operation on each first face image according to the average coordinate of each key point by a moving least square method.
In one possible design, after the sending the target image to the terminal device, the method further includes:
acquiring an editing instruction sent by the terminal equipment, wherein the editing instruction comprises target special effect information;
and editing the target image according to the target special effect information.
According to the image processing method provided by the embodiment, the editing instruction sent by the terminal device is obtained, and the target image is edited according to the target special effect information in the editing instruction. Thereby improving the effect of face editing processing.
In one possible design, the target effect information includes at least one of a face adjustment effect, a hair style adjustment effect, and a sticker effect.
In a possible design, the image cropping instruction is generated by clicking an image cropping icon preset on the display interface of the terminal device by a user.
A second aspect of the present invention provides an image processing apparatus comprising:
the system comprises an acquisition module, a display module and a display module, wherein the acquisition module is used for acquiring an image cutting instruction sent by terminal equipment, and the image cutting instruction comprises a face image to be cut;
the determining module is used for determining a target transformation affine matrix according to at least one key point corresponding to a preset average face image and the face image;
the cutting module is used for carrying out cutting operation on the face image according to the target transformation affine matrix to obtain a cut target image;
and the sending module is used for sending the target image to the terminal equipment so as to enable a user to edit the target image.
A third aspect of the present invention provides an image processing apparatus comprising: a memory, a processor;
a memory; a memory for storing the processor-executable instructions;
wherein the processor is configured to perform the image processing method according to the first aspect by the processor.
A fourth aspect of the present invention provides a computer-readable storage medium having stored thereon computer-executable instructions for implementing the image processing method according to the first aspect when the computer-executable instructions are executed by a processor.
According to the image processing method, the image processing device, the image processing equipment and the computer readable storage medium, the target transformation affine matrix is determined according to at least one average key point corresponding to a preset average face according to the image cutting instruction sent by the terminal equipment, the cutting operation is carried out on the face image to be cut according to the target transformation affine matrix, and the cut target image is sent to the terminal equipment to be displayed. The average face is generated according to a large number of face images, so that the facial features of most people can be represented, the face images are cut according to the target transformation affine matrix determined by the average face, the face cutting precision can be effectively improved, the effect of special effect processing according to the cut target images is better, the image processing requirements of users can be better met, and the user experience is improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings can be obtained by those skilled in the art according to the drawings.
FIG. 1 is a schematic diagram of a network architecture on which the present invention is based;
fig. 2 is a schematic flowchart of an image processing method according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a display interface provided in an embodiment of the invention;
FIG. 4 is a schematic diagram of an average face according to an embodiment of the present invention;
fig. 5 is a schematic flowchart of an image processing method according to a second embodiment of the present invention;
fig. 6 is a schematic flowchart of an image processing method according to a third embodiment of the present invention;
fig. 7 is a schematic structural diagram of an image processing apparatus according to a fourth embodiment of the present invention;
fig. 8 is a schematic structural diagram of an image processing apparatus according to a fifth embodiment of the present invention;
fig. 9 is a schematic structural diagram of an image processing apparatus according to a sixth embodiment of the present invention;
fig. 10 is a schematic structural diagram of an image processing apparatus according to a seventh embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other examples obtained based on the examples in the present invention are within the scope of the present invention.
The invention provides an image processing method, an image processing device, image processing equipment and a computer readable storage medium, aiming at the technical problem that the face area cutting precision is not high and the subsequent special effect processing effect is not good in the image processing method.
It should be noted that the image processing method, apparatus, device and computer-readable storage medium provided in the present application may be applied in various image processing scenarios.
The inventor finds that similar features generally exist on a human face through research, can summarize human face features corresponding to a large number of human face images to generate an average face image, and then performs human face region clipping according to the average face image.
The inventor further researches and discovers that when an image clipping instruction sent by a user through terminal equipment is obtained, a target transformation affine matrix can be determined by adopting at least one average key point corresponding to a preset average face and a face image to be clipped. And then the human face image can be cut according to the target transformation affine matrix. In order to enable the user to perform an editing operation on the cropped image, the cropped target image may be transmitted to the terminal device. The average face is generated according to a large number of face images, so that the facial features of most people can be represented, the face images are cut according to the target transformation affine matrix determined by the average face, the face cutting precision can be effectively improved, the effect of special effect processing according to the cut target images is better, the image processing requirements of users can be better met, and the user experience is improved.
Fig. 1 is a schematic diagram of a network architecture based on the present invention, and as shown in fig. 1, the network architecture based on the present invention at least includes: a terminal device 1, an image processing apparatus 2, and a data server 3. Wherein, the image processing device 2 is written by C/C + +, Java, Shell or Python; the terminal device 1 may be a desktop computer, a tablet computer, or the like. The data server 3 may be a cloud server or a server cluster, and a large amount of data is stored therein. Specifically, the image processing apparatus 2 is communicatively connected to the terminal device 1 and the data server 3, and is capable of exchanging information with the terminal device 1 and the data server 3.
Fig. 2 is a schematic flowchart of an image processing method according to an embodiment of the present invention, and as shown in fig. 2, the method includes:
The execution subject of the embodiment is an image processing device, and the image processing device is in communication connection with the data server, so that data interaction can be performed with the data server. The image processing apparatus may be provided in a terminal device of a target user, or may be an apparatus independent from the terminal device, and is further connected to the terminal device in a communication manner, so that information interaction with the terminal device is possible.
In the present embodiment, when a user edits an image on a terminal device, in order to improve the effect of image editing, it is first necessary to perform a cropping operation on a face of the person in the image. Specifically, the user can generate the image cropping instruction by triggering an image cropping icon set on the display interface of the terminal device. Fig. 3 is a schematic view of a display interface provided in an embodiment of the present invention, and as shown in fig. 3, a user may trigger the image cropping icon to generate a corresponding image cropping instruction. The user can trigger the image cropping icon in any one of a single-click mode, a double-click mode, a long-press mode, a dragging mode and the like, and the method is not limited in this respect. Accordingly, the image processing device can obtain an image cropping instruction sent by the terminal equipment, wherein the image cropping instruction comprises a human face image to be cropped.
And 102, determining a target transformation affine matrix according to at least one key point corresponding to a preset average face image and the face image.
In this embodiment, after acquiring the image cropping instruction sent by the terminal device, the image processing apparatus may acquire a pre-stored average face image from the data server, where the average face image is obtained by performing average calculation based on a large number of face images and has feature information of most faces. Fig. 4 is a schematic diagram of an average face according to an embodiment of the present invention, as shown in fig. 4, the average face image includes at least one key point, where the key point is an average value of corresponding key points in multiple face images, and the key point may include an eye key point, a nose key point, an eyebrow key point, a lip key point, and the like. After the average face image is acquired, a target transformation affine matrix can be determined according to at least one key point in the average face image and the face image to be cut.
Specifically, on the basis of the first embodiment, a target transformation affine matrix may be specifically determined according to a preset transformation formula, at least one key point corresponding to a preset average face image, and the face image, where the transformation formula is as follows:
u＝a1x+b1y+c1
v＝a2x+b2y+c2
wherein, (x, y) is the coordinate of any one point in the target key points in the face image, (u, v) is the average coordinate of the key points corresponding to the target key in the average face image, and the matrix where abc is located is marked as M and is the target affine transformation matrix to be solved. According to the formula, one equation can be constructed for each point, in practical application, one equation can be constructed for each key point, and therefore a plurality of equations can be constructed and jointly solved, and the optimal target affine transformation matrix can be obtained.
The M matrix form is as follows:
and 103, performing cutting operation on the face image according to the target transformation affine matrix to obtain a cut target image.
In this embodiment, after determining the target transformed affine matrix according to at least one key point in the average face image and the face image to be clipped, the clipping operation may be performed on the face image according to the target transformed affine matrix to obtain the clipped target image. The average face is generated according to a large number of face images, so that the facial features of most people can be represented, the face images are cut according to the target transformation affine matrix determined by the average face, and the face cutting precision can be effectively improved.
And 104, sending the target image to the terminal equipment so that a user can edit the target image.
In the present embodiment, in order to enable the user to perform editing processing on the clipped image, the clipped target image may be transmitted to the terminal device after being obtained. Accordingly, after the terminal device acquires the target image, the terminal device can display the target image on the display interface. The user can edit the target image on the terminal equipment according to the self requirement.
According to the image processing method provided by the embodiment, a target transformation affine matrix is determined according to at least one average key point corresponding to a preset average face according to an image clipping instruction sent by the terminal device, a clipping operation is performed on a face image to be clipped according to the target transformation affine matrix, and the clipped target image is sent to the terminal device to be displayed. The average face is generated according to a large number of face images, so that the facial features of most people can be represented, the face images are cut according to the target transformation affine matrix determined by the average face, the face cutting precision can be effectively improved, the effect of special effect processing according to the cut target images is better, the image processing requirements of users can be better met, and the user experience is improved.
Fig. 5 is a schematic flowchart of an image processing method according to a second embodiment of the present invention, and on the basis of the first embodiment, as shown in fig. 5, before step 102, the method further includes:
In the present embodiment, in order to be able to determine the target transformed affine matrix from the average face image, it is first necessary to obtain the average face image. Specifically, a preset first face set may be obtained from the data server, where the first face set includes a plurality of first face images, and each of the first face images has a consistent size and includes at least one key point. And respectively determining the coordinate information of each key point in each first face image aiming at each first face image, and further calculating the average coordinate of each key point of a plurality of first face images in the first face set to obtain the average coordinate of at least one key point.
Further, image deformation may be performed on the first face image in the first face image set according to the average coordinate of at least one key point, so that the distance between the coordinate of the key point extracted for each object and the corresponding average coordinate in the first face image is smaller than or equal to a preset distance, and then the deformed face image set is obtained. It is understood that after the first face image is deformed, the deformed first face image is obtained.
Specifically, for a first face image in the first set of face images: firstly, the execution main body can deform a first face image according to a transformation matrix input by a user to obtain a deformed first face image; then, whether the distance between the coordinate of the key point extracted aiming at each object and the corresponding average coordinate in the deformed first face image is smaller than or equal to the preset distance or not can be determined; if the distance is smaller than or equal to the preset distance, obtaining a deformed first face image; if the distance is greater than the preset distance, the first face image can be deformed according to the transformation matrix input by the user again until the distance between the coordinate of the key point extracted aiming at each object and the corresponding average coordinate in the deformed first face image is less than or equal to the preset distance.
Specifically, on the basis of any of the above embodiments, step 204 includes:
and performing deformation operation on each first face image according to the average coordinate of each key point by a moving least square method.
In this embodiment, the image deformation may be performed on the first face image in the first face image set by using a Moving Least Square (MLS) method. The executing body may perform image transformation on the first face image by using another image transformation algorithm. It should be noted that, by moving the least square method, the first face image is deformed, so that the situation that the user inputs the transformation matrix for many times can be avoided, and the time for deforming the first face image is further shortened.
In this embodiment, after obtaining the set of deformed face images, an average face image of the set of deformed first face images may be further determined. And the pixel value of each pixel point in the average face image is the average value of the pixel values of the corresponding pixel points in the deformed first face image set.
Specifically, the image may be essentially regarded as a pixel value matrix formed by pixel values of the pixel points, that is, each deformed first face image may be regarded as a pixel value matrix. In this way, the image processing apparatus can average the matrix of pixel values corresponding to the set of deformed face images. It is understood that the pixel value matrix obtained after averaging is the above average face image. The above-described averaging of the pixel value matrices may be averaging of pixel values at the same position in the pixel value matrix.
Further, on the basis of any of the above embodiments, before step 201, the method further includes:
acquiring a preset second face image set;
and cutting the second face image with the key points in the second face image set according to a preset target size to obtain the first face image set.
In this embodiment, the second face images with the target number and extracted key points may be selected from the second face image set. The target number may be preset or determined according to actual requirements. In order to facilitate subsequent operations, each second face image can be cut according to a preset target size, and a first face image set is obtained. The target size may be obtained according to an empirical value, or may be set by the user, which is not limited in the present invention.
In the image processing method provided by this embodiment, the average coordinates of the key points indicating the same object are determined by the coordinates of the extracted key points for each first face image in the first face image set, and on the basis of the obtained average coordinates, the first face image is subjected to image deformation to obtain the deformed first face image set, thereby generating the average face image of the deformed first face image set. In the process of generating the average face image, triangulation on each first face image is not needed, so that the calculation amount of a server is reduced, and a basis is provided for the generation of a subsequent target affine transformation matrix.
Fig. 6 is a schematic flowchart of an image processing method according to a third embodiment of the present invention, and on the basis of any of the foregoing embodiments, as shown in fig. 6, after step 104, the method further includes:
and 302, editing the target image according to the target special effect information.
In the present embodiment, after the clipped target image is obtained, the target image may be transmitted to the terminal device. Accordingly, after the terminal device acquires the target image, the terminal device can display the target image on the display interface. The user can edit the target image on the terminal equipment according to the self requirement. Specifically, the user may generate an editing instruction on the terminal device, where the editing instruction includes the target special effect information. The target special effect information includes at least one of a face adjustment special effect, a hair style adjustment special effect, and a sticker special effect. After the editing instruction is acquired, the target image can be edited according to the target special effect information.
In terms of the practical application distance, a user can perform face thinning operation on the face in the target image, and the target image is obtained after being cut according to the target transformation affine matrix, so that the accuracy is high, and the corresponding editing operation effect according to the target image is good.
According to the image processing method provided by the embodiment, the editing instruction sent by the terminal device is obtained, and the target image is edited according to the target special effect information in the editing instruction. Thereby improving the effect of face editing processing.
Fig. 7 is a schematic structural diagram of an image processing apparatus according to a fourth embodiment of the present invention, and as shown in fig. 7, the apparatus includes: the image cropping system comprises an acquisition module 41, a determination module 42, a cropping module 43 and a sending module 44, wherein the acquisition module 41 is configured to acquire an image cropping instruction sent by a terminal device, and the image cropping instruction includes a face image to be cropped; a determining module 42, configured to determine a target transformation affine matrix according to at least one key point corresponding to a preset average face image and the face image; the cutting module 43 is configured to perform a cutting operation on the face image according to the target transformed affine matrix, so as to obtain a cut target image; a sending module 44, configured to send the target image to the terminal device, so that the user performs editing processing on the target image.
Further, on the basis of the fourth embodiment, the determining module is configured to:
determining a target transformation affine matrix according to a preset transformation formula, at least one key point corresponding to a preset average face image and the face image, wherein the transformation formula is as follows:
u＝a1x+b1y+c1
v＝a2x+b2y+c2
wherein, (x, y) is the coordinate of any one point in the target key points in the face image, and (u, v) is the average coordinate of the key points corresponding to the target key points in the average face image.
Further, on the basis of the fourth embodiment, the image cropping command is generated by clicking an image cropping icon preset on the display interface of the terminal device by the user.
The image processing apparatus provided in this embodiment determines, according to an image clipping instruction sent by a terminal device, a target transformation affine matrix according to at least one average key point corresponding to a preset average face, performs a clipping operation on a face image to be clipped according to the target transformation affine matrix, and sends the clipped target image to the terminal device for display. The average face is generated according to a large number of face images, so that the facial features of most people can be represented, the face images are cut according to the target transformation affine matrix determined by the average face, the face cutting precision can be effectively improved, the effect of special effect processing according to the cut target images is better, the image processing requirements of users can be better met, and the user experience is improved.
Fig. 8 is a schematic structural diagram of an image processing apparatus according to a fifth embodiment of the present invention, and based on the fourth embodiment, as shown in fig. 8, the apparatus further includes: the image processing device comprises a set acquisition module 51, a coordinate determination module 52, a calculation module 53, a deformation module 54 and a processing module 55, wherein the set acquisition module 51 is used for acquiring a preset first face set, and first face images in the first face set are consistent in size and all include at least one key point; a coordinate determination module 52, configured to determine at least one key point coordinate corresponding to each first face image in the first face combination; a calculating module 53, configured to calculate, according to at least one coordinate of a key point corresponding to each first face image, an average coordinate of each key point corresponding to the first face set; a deformation module 54, configured to perform a deformation operation on each first face image according to the average coordinate of each key point to obtain a deformed first face image, where a distance between each key point coordinate in the deformed first face image and the average coordinate is smaller than a preset threshold; and the processing module 55 is configured to determine an average face image corresponding to the deformed first face image set, where a pixel value of each pixel in the average face image is an average value of pixel values of corresponding pixels in each deformed first face image in the deformed first face image set.
The image processing apparatus according to this embodiment determines average coordinates of key points indicating the same object by using coordinates of key points extracted for each first face image in the first face image set, obtains a first face image set after transformation by performing image transformation on the first face image on the basis of the obtained average coordinates, and generates an average face image of the first face image set after transformation. In the process of generating the average face image, triangulation on each first face image is not needed, so that the calculation amount of a server is reduced, and a basis is provided for the generation of a subsequent target affine transformation matrix.
Further, on the basis of any of the above embodiments, the deformation module is configured to:
and performing deformation operation on each first face image according to the average coordinate of each key point by a moving least square method.
Further, on the basis of any of the above embodiments, the set obtaining module is configured to:
acquiring a preset second face image set;
and cutting the second face image with the key points in the second face image set according to a preset target size to obtain the first face image set.
Fig. 9 is a schematic structural diagram of an image processing apparatus according to a sixth embodiment of the present invention, and based on any of the foregoing embodiments, as shown in fig. 9, the apparatus further includes: the terminal equipment comprises an instruction acquisition module 61 and an editing module 62, wherein the instruction acquisition module 61 is used for acquiring an editing instruction sent by the terminal equipment, and the editing instruction comprises target special effect information; and the editing module 62 is configured to perform an editing operation on the target image according to the target special effect information.
Further, on the basis of any of the above embodiments, the target special effect information includes at least one of a face adjustment special effect, a hair style adjustment special effect, and a sticker special effect.
The image processing apparatus provided in this embodiment, by acquiring an editing instruction sent by a terminal device, performs an editing operation on a target image according to target special effect information in the editing instruction. Thereby improving the effect of face editing processing.
Fig. 10 is a schematic structural diagram of an image processing apparatus according to a seventh embodiment of the present invention, and as shown in fig. 10, the image processing apparatus includes: a memory 71, a processor 72;
a memory 71; a memory 71 for storing instructions executable by the processor 72;
wherein the processor 72 is configured to execute the image processing method according to any of the above embodiments by the processor 72.
The memory 71 stores programs. In particular, the program may include program code comprising computer operating instructions. The memory 71 may comprise a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.
The processor 72 may be a Central Processing Unit (CPU), an Application Specific Integrated Circuit (ASIC), or one or more Integrated circuits configured to implement embodiments of the present invention.
Alternatively, in a specific implementation, if the memory 71 and the processor 72 are implemented independently, the memory 71 and the processor 72 may be connected to each other through a bus and perform communication with each other. The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, an Extended ISA (EISA) bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown in FIG. 10, but this is not intended to represent only one bus or type of bus.
Alternatively, in a specific implementation, if the memory 71 and the processor 72 are integrated on a chip, the memory 71 and the processor 72 may perform the same communication through an internal interface.
The invention further provides a computer-readable storage medium, in which computer-executable instructions are stored, and when the computer-executable instructions are executed by a processor, the computer-readable storage medium is used for implementing the image processing method according to any one of the above embodiments.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working process of the apparatus described above may refer to the corresponding process in the foregoing method embodiment, and is not described herein again.
Those of ordinary skill in the art will understand that: all or a portion of the steps of implementing the above-described method embodiments may be performed by hardware associated with program instructions. The program may be stored in a computer-readable storage medium. When executed, the program performs steps comprising the method embodiments described above; and the aforementioned storage medium includes: various media that can store program codes, such as ROM, RAM, magnetic or optical disks.
Finally, it should be noted that: the above embodiments are only used to illustrate the technical solution of the present invention, and not to limit the same; while the invention has been described in detail and with reference to the foregoing embodiments, it will be understood by those skilled in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present invention.
Claims (16)
1. An image processing method, comprising:
acquiring an image cutting instruction sent by terminal equipment, wherein the image cutting instruction comprises a face image to be cut;
determining a target transformation affine matrix according to at least one key point corresponding to a preset average face image and the face image;
cutting the face image according to the target transformation affine matrix to obtain a cut target image;
sending the target image to the terminal equipment so that a user can edit the target image;
the determining of the target transformation affine matrix according to at least one key point corresponding to a preset average face image and the face image comprises the following steps:
determining a target transformation affine matrix according to a preset transformation formula, at least one key point corresponding to a preset average face image and the face image, wherein the transformation formula is as follows:
u＝a1x+b1y+c1
v＝a2x+b2y+c2
wherein, (x, y) is the coordinate of any one point in the target key points in the face image, and (u, v) is the average coordinate of the key points corresponding to the target key points in the average face image.
2. The method according to claim 1, wherein before determining the target transformed affine matrix according to at least one key point corresponding to a preset average face image and the face image, the method further comprises:
acquiring a preset first face set, wherein first face images in the first face set are consistent in size and all comprise at least one key point;
determining at least one key point coordinate corresponding to each first face image in the first face set;
calculating the average coordinate of each key point corresponding to the first face set according to at least one key point coordinate corresponding to each first face image;
performing deformation operation on each first face image according to the average coordinate of each key point to obtain a deformed first face image, wherein the distance between each key point coordinate in the deformed first face image and the average coordinate is smaller than a preset threshold value;
and determining an average face image corresponding to the deformed first face image set, wherein the pixel value of each pixel point in the average face image is the average value of the pixel values of corresponding pixel points in each deformed first face image in the deformed first face image set.
3. The method of claim 2, wherein the obtaining a preset first set of faces comprises:
acquiring a preset second face image set;
and cutting the second face image with the key points in the second face image set according to a preset target size to obtain the first face image set.
4. The method according to claim 2, wherein the warping of each of the first face images according to the average coordinates of the key points comprises:
and performing deformation operation on each first face image according to the average coordinate of each key point by a moving least square method.
5. The method according to any one of claims 1-4, wherein after sending the target image to the terminal device, further comprising:
acquiring an editing instruction sent by the terminal equipment, wherein the editing instruction comprises target special effect information;
and editing the target image according to the target special effect information.
6. The method of claim 5, wherein the target effect information includes at least one of a face adjustment effect, a hair style adjustment effect, and a sticker effect.
7. The method according to any one of claims 1 to 4, wherein the image cropping instruction is generated by clicking an image cropping icon preset on the display interface of the terminal device by a user.
8. An image processing apparatus characterized by comprising:
the system comprises an acquisition module, a display module and a display module, wherein the acquisition module is used for acquiring an image cutting instruction sent by terminal equipment, and the image cutting instruction comprises a face image to be cut;
the determining module is used for determining a target transformation affine matrix according to at least one key point corresponding to a preset average face image and the face image;
the cutting module is used for carrying out cutting operation on the face image according to the target transformation affine matrix to obtain a cut target image;
the sending module is used for sending the target image to the terminal equipment so as to enable a user to edit the target image;
the determination module is to:
determining a target transformation affine matrix according to a preset transformation formula, at least one key point corresponding to a preset average face image and the face image, wherein the transformation formula is as follows:
u＝a1x+b1y+c1
v＝a2x+b2y+c2
wherein, (x, y) is the coordinate of any one point in the target key points in the face image, and (u, v) is the average coordinate of the key points corresponding to the target key points in the average face image.
9. The apparatus of claim 8, further comprising:
the system comprises a set acquisition module, a first face acquisition module and a second face acquisition module, wherein the set acquisition module is used for acquiring a preset first face set, and first face images in the first face set are consistent in size and all comprise at least one key point;
the coordinate determination module is used for determining at least one key point coordinate corresponding to each first face image in the first face set;
the calculation module is used for calculating the average coordinate of each key point corresponding to the first face set according to at least one key point coordinate corresponding to each first face image;
the deformation module is used for carrying out deformation operation on each first face image according to the average coordinate of each key point to obtain a deformed first face image, wherein the distance between each key point coordinate in the deformed first face image and the average coordinate is smaller than a preset threshold value;
and the processing module is used for determining an average face image corresponding to the deformed first face image set, wherein the pixel value of each pixel point in the average face image is the average value of the pixel values of corresponding pixel points in each deformed first face image in the deformed first face image set.
10. The apparatus of claim 9, wherein the set acquisition module is configured to:
acquiring a preset second face image set;
and cutting the second face image with the key points in the second face image set according to a preset target size to obtain the first face image set.
11. The apparatus of claim 9, wherein the deformation module is to:
and performing deformation operation on each first face image according to the average coordinate of each key point by a moving least square method.
12. The apparatus according to any one of claims 8-11, further comprising:
the instruction acquisition module is used for acquiring an editing instruction sent by the terminal equipment, wherein the editing instruction comprises target special effect information;
and the editing module is used for editing the target image according to the target special effect information.
13. The apparatus of claim 12, wherein the target effect information comprises at least one of a face adjustment effect, a hair style adjustment effect, and a sticker effect.
14. The apparatus according to any one of claims 8-11, wherein the image cropping instruction is generated by a user clicking an image cropping icon preset on the display interface of the terminal device.
15. An image processing apparatus characterized by comprising: a memory, a processor;
a memory; a memory for storing the processor-executable instructions;
wherein the processor is configured to perform the image processing method of any one of claims 1-7 by the processor.
16. A computer-readable storage medium having computer-executable instructions stored therein, which when executed by a processor, are configured to implement the image processing method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010402093.6A CN111583280B (en) | 2020-05-13 | 2020-05-13 | Image processing method, device, equipment and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010402093.6A CN111583280B (en) | 2020-05-13 | 2020-05-13 | Image processing method, device, equipment and computer readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111583280A CN111583280A (en) | 2020-08-25 |
| CN111583280B true CN111583280B (en) | 2022-03-15 |
Family
ID=72112111
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010402093.6A Active CN111583280B (en) | 2020-05-13 | 2020-05-13 | Image processing method, device, equipment and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111583280B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112508773B (en) * | 2020-11-20 | 2024-02-09 | 小米科技(武汉)有限公司 | Image processing method and device, electronic equipment and storage medium |
| CN112766215A (en) * | 2021-01-29 | 2021-05-07 | 北京字跳网络技术有限公司 | Face fusion method and device, electronic equipment and storage medium |
| CN113643417B (en) * | 2021-08-17 | 2023-06-27 | 腾讯科技（深圳）有限公司 | Image adjustment method, device, electronic equipment and storage medium |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105787878B (en) * | 2016-02-25 | 2018-12-28 | 杭州格像科技有限公司 | A kind of U.S. face processing method and processing device |
| CN106295567B (en) * | 2016-08-10 | 2019-04-12 | 腾讯科技（深圳）有限公司 | A kind of localization method and terminal of key point |
| CN106909888B (en) * | 2017-01-22 | 2021-02-05 | 南京开为网络科技有限公司 | Face key point tracking system and method applied to mobile equipment terminal |
| CN109685740B (en) * | 2018-12-25 | 2023-08-11 | 努比亚技术有限公司 | Face correction method and device, mobile terminal and computer readable storage medium |
| CN110189252B (en) * | 2019-06-10 | 2023-08-18 | 北京字节跳动网络技术有限公司 | Method and device for generating average face image |
-
2020
- 2020-05-13 CN CN202010402093.6A patent/CN111583280B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN111583280A (en) | 2020-08-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111583280B (en) | Image processing method, device, equipment and computer readable storage medium | |
| JP6871416B2 (en) | Methods and devices for determining facial image quality, electronics and computer storage media | |
| CN108062526B (en) | Human body posture estimation method and mobile terminal | |
| CN109922355B (en) | Live virtual image broadcasting method, live virtual image broadcasting device and electronic equipment | |
| CN109598744B (en) | Video tracking method, device, equipment and storage medium | |
| CN113420719B (en) | Method and device for generating motion capture data, electronic equipment and storage medium | |
| CN111667504B (en) | Face tracking method, device and equipment | |
| CN111027438A (en) | Human body posture migration method, mobile terminal and computer storage medium | |
| CN112381071A (en) | Behavior analysis method of target in video stream, terminal device and medium | |
| CN112330527A (en) | Image processing method, image processing apparatus, electronic device, and medium | |
| US20230237777A1 (en) | Information processing apparatus, learning apparatus, image recognition apparatus, information processing method, learning method, image recognition method, and non-transitory-computer-readable storage medium | |
| CN111429338A (en) | Method, apparatus, device and computer-readable storage medium for processing video | |
| CN110867243B (en) | Image annotation method, device, computer system and readable storage medium | |
| CN110189252B (en) | Method and device for generating average face image | |
| JP2019517079A (en) | Shape detection | |
| CN113327320A (en) | Three-dimensional model reconstruction method, device, equipment and computer readable storage medium | |
| KR101791604B1 (en) | Method and apparatus for estimating position of head, computer readable storage medium thereof | |
| EP3699865B1 (en) | Three-dimensional face shape derivation device, three-dimensional face shape deriving method, and non-transitory computer readable medium | |
| CN111461971B (en) | Image processing method, device, equipment and computer readable storage medium | |
| CN112929562B (en) | Video jitter processing method, device, equipment and storage medium | |
| CN105631938B (en) | Image processing method and electronic equipment | |
| CN112714337A (en) | Video processing method and device, electronic equipment and storage medium | |
| CN117197855A (en) | Face key point labeling method and device, electronic equipment and storage medium | |
| US11127218B2 (en) | Method and apparatus for creating augmented reality content | |
| US11481940B2 (en) | Structural facial modifications in images |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |