CN111461971B - Image processing method, device, equipment and computer readable storage medium - Google Patents
Image processing method, device, equipment and computer readable storage medium Download PDFInfo
- Publication number
- CN111461971B CN111461971B CN202010423985.4A CN202010423985A CN111461971B CN 111461971 B CN111461971 B CN 111461971B CN 202010423985 A CN202010423985 A CN 202010423985A CN 111461971 B CN111461971 B CN 111461971B
- Authority
- CN
- China
- Prior art keywords
- image
- face
- face image
- age
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 30
- 238000012545 processing Methods 0.000 claims abstract description 99
- 238000006243 chemical reaction Methods 0.000 claims abstract description 37
- 238000000034 method Methods 0.000 claims abstract description 32
- 230000000694 effects Effects 0.000 claims description 41
- 230000001815 facial effect Effects 0.000 claims description 14
- 238000012549 training Methods 0.000 claims description 11
- 238000010586 diagram Methods 0.000 description 17
- 230000008569 process Effects 0.000 description 10
- 230000009466 transformation Effects 0.000 description 9
- 238000013461 design Methods 0.000 description 8
- 239000011159 matrix material Substances 0.000 description 8
- 238000004891 communication Methods 0.000 description 5
- 238000012935 Averaging Methods 0.000 description 3
- 230000003042 antagnostic effect Effects 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 3
- 230000008859 change Effects 0.000 description 3
- 238000013528 artificial neural network Methods 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 238000003062 neural network model Methods 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000011022 operating instruction Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/04—Context-preserving transformations, e.g. by using an importance map
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
Abstract
The present disclosure provides an image processing method, apparatus, device, and computer-readable storage medium, the method comprising: acquiring an image processing instruction sent by terminal equipment, wherein the image processing instruction comprises a first face image to be processed, and the age of a face in the face image is greater than a preset first age threshold; inputting the face image into a preset age conversion model according to an image processing instruction to obtain a second face image, wherein the age of the face in the second face image is smaller than a preset second age threshold, and a preset age difference value exists between the first age threshold and the second age threshold; editing the first face image according to the second face image to obtain a target image; and sending the target image to the terminal equipment. Be different from the operation that directly increases the sticker to the image that image acquisition device gathered among the prior art, adjust through the age to people's face in the first face image to first face image changes great, and is interesting stronger, and then can improve user experience.
Description
Technical Field
The present disclosure relates to the field of image processing, and in particular, to an image processing method, an image processing apparatus, an image processing device, and a computer-readable storage medium.
Background
With the development of science and technology, more and more application software goes into the life of users, and the amateur life of the users is gradually enriched, such as short video APP and the like. The user can record life by adopting modes of videos, photos and the like and upload the life on the short-time video APP. However, the type of the video or the photo shot only by the image capturing device of the terminal device is single, which results in poor user experience.
In order to improve user experience, the existing short video APP generally performs special effect processing on a video uploaded by a user, for example, a sticker may be added on a face of the user. However, the change of the video or the picture is often small only by adding operations such as sticker on the basis of the video or the picture shot by the image acquisition device of the terminal equipment, so that the effect is single and the interestingness is not strong.
Disclosure of Invention
The present disclosure provides an image processing method, an image processing apparatus, an image processing device, and a computer-readable storage medium, which are used to solve the technical problems that the change of video or picture is small, the effect is single, and the interestingness is not strong in the existing image processing.
A first aspect of the present disclosure is to provide an image processing method, including:
acquiring an image processing instruction sent by terminal equipment, wherein the image processing instruction comprises a first face image to be processed, and the age of a face in the face image is greater than a preset first age threshold;
inputting the face image into a preset age conversion model according to the image processing instruction to obtain a second face image, wherein the age of the face in the second face image is smaller than a preset second age threshold, and a preset age difference value exists between the first age threshold and the second age threshold;
editing the first face image according to the second face image to obtain a target image;
and sending the target image to the terminal equipment so as to enable a user to edit the target image.
In one possible design, the editing the first facial image according to the second facial image to obtain a target image includes:
determining a set of deformation of the second facial image relative to the first facial image;
and editing the first face image according to the deformation array and a target mask corresponding to a preset average face image to obtain the target image.
In one possible design, the value corresponding to the target mask is between 0 and 1, and the value of the target mask is in direct proportion to the probability that the pixel point belongs to the face region;
correspondingly, the editing operation of the first face image according to the deformation array and a target mask corresponding to a preset average face image includes:
and multiplying the deformation array by the numerical value of the target mask to obtain the target image.
In one possible design, the first face image is obtained by cropping according to a preset average face image.
In one possible design, the method further includes:
acquiring a preset third face set, wherein the sizes of third face images in the third face set are consistent and each third face image comprises at least one key point;
determining at least one key point coordinate corresponding to each third face image in the third face set;
calculating the average coordinate of each key point corresponding to the third face set according to at least one key point coordinate corresponding to each third face image;
performing deformation operation on each third face image according to the average coordinate of each key point to obtain a deformed third face image, wherein the distance between each key point coordinate in the deformed third face image and the average coordinate is smaller than a preset threshold value;
and determining an average face image corresponding to the deformed third face image set, wherein the pixel value of each pixel point in the average face image is the average value of the pixel values of corresponding pixel points in the deformed third face image set.
In one possible design, the age transformation model includes a generator and an arbiter;
correspondingly, before the inputting the face image into a preset age conversion model according to the image processing instruction, the method further comprises:
acquiring a data set to be trained from a data server, wherein the data set to be trained comprises a plurality of first face images and second face images corresponding to the first face images;
inputting the first face image into a generator to obtain a generated image corresponding to the first face image;
and inputting the generated image and the second face image into the discriminator so that the discriminator judges the authenticity of the generated image and the second face image, and performing supervision training on the generator according to a judgment result until the model converges to obtain the age conversion model.
In one possible design, after the sending the target image to the terminal device, the method further includes:
acquiring an image editing instruction sent by terminal equipment, wherein the image editing instruction comprises target special effect information;
and editing the target image according to the target special effect information.
In one possible design, the target effect information includes at least one of a face adjustment effect, a hair style adjustment effect, and a sticker effect.
In a possible design, the image processing instruction is generated by a user clicking a preset image processing icon on the display interface of the terminal device.
A second aspect of the present disclosure is to provide an image processing apparatus comprising:
the terminal equipment comprises an acquisition module, a processing module and a processing module, wherein the acquisition module is used for acquiring an image processing instruction sent by the terminal equipment, the image processing instruction comprises a first face image to be processed, and the age of a face in the face image is greater than a preset first age threshold value;
the conversion module is used for inputting the face image into a preset age conversion model according to the image processing instruction to obtain a second face image, wherein the age of the face in the second face image is smaller than a preset second age threshold, and a preset age difference value exists between the first age threshold and the second age threshold;
the editing module is used for carrying out editing operation on the first face image according to the second face image to obtain a target image;
and the sending module is used for sending the target image to the terminal equipment so as to enable a user to edit the target image.
A third aspect of the present disclosure is to provide an image processing apparatus comprising: a memory, a processor;
a memory; a memory for storing the processor-executable instructions;
wherein the processor is configured to perform the image processing method according to the first aspect by the processor.
A fourth aspect of the present disclosure is to provide a computer-readable storage medium having stored therein computer-executable instructions for implementing the image processing method according to the first aspect when the computer-executable instructions are executed by a processor.
According to the image processing method, the image processing device, the image processing equipment and the computer readable storage medium, the first face image is input into the preset age conversion model according to the image processing instruction sent by the terminal equipment, and the second face image with the face age smaller than the second age threshold is obtained. Therefore, subsequent editing operation can be carried out on the first face image according to the second face image, and the target image with the face age smaller than the second age threshold value is obtained. And sending the target face image to the terminal equipment, so that the user can process the age-converted target image on the terminal equipment. Be different from the operation that directly increases the sticker to the image that image acquisition device gathered among the prior art, adjust through the age to people's face in the first face image to first face image changes great, and is interesting stronger, and then can improve user experience.
Drawings
In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present disclosure, and other drawings can be obtained by those skilled in the art according to the drawings.
FIG. 1 is a schematic diagram of a network architecture upon which the present disclosure is based;
fig. 2 is a schematic flowchart of an image processing method according to a first embodiment of the disclosure;
FIG. 3 is a schematic diagram of a display interface provided by an embodiment of the present disclosure;
fig. 4 is a schematic flowchart of an image processing method according to a second embodiment of the disclosure;
FIG. 5 is a block diagram of yet another system upon which the present disclosure is based;
fig. 6 is a schematic flowchart of an image processing method according to a third embodiment of the present disclosure;
FIG. 7 is a network architecture diagram of an age translation model provided by an embodiment of the present disclosure;
fig. 8 is a schematic flowchart of an image processing method according to a fourth embodiment of the disclosure;
fig. 9 is a schematic structural diagram of an image processing apparatus according to a fifth embodiment of the present disclosure;
fig. 10 is a schematic structural diagram of an image processing apparatus according to a sixth embodiment of the present disclosure;
fig. 11 is a schematic structural diagram of an image processing apparatus according to a seventh embodiment of the present disclosure;
fig. 12 is a schematic structural diagram of an image processing apparatus according to an eighth embodiment of the present disclosure;
fig. 13 is a schematic structural diagram of an image processing apparatus according to a ninth embodiment of the present disclosure.
Detailed Description
To make the objects, technical solutions and advantages of the embodiments of the present disclosure more apparent, the technical solutions in the embodiments of the present disclosure will be described clearly and completely with reference to the drawings in the embodiments of the present disclosure, and it is obvious that the described embodiments are some, but not all embodiments of the present disclosure. All other embodiments obtained based on the embodiments in the disclosure belong to the protection scope of the disclosure.
In view of the above-mentioned technical problems that the conventional image processing has small changes to videos or pictures, so that the effect is single and the interestingness is not strong, the present disclosure provides an image processing method, an image processing device, an image processing apparatus, and a computer-readable storage medium.
It should be noted that the image processing method, apparatus, device and computer readable storage medium provided in the present application may be applied to various image processing scenarios.
The existing image processing method is generally to directly add stickers to images or videos collected by an image collecting device arranged on terminal equipment, and the change of the images is small, so that the visual effect is poor, and the user experience is poor.
In the face of the problems in the prior art, the inventor finds through research that the age of the face in the face image acquired by the image acquisition device can be adjusted in the image processing process, and operations such as adding stickers are performed on the basis of age adjustment, so that the interestingness of the image can be greatly improved.
The inventor further researches and discovers that according to an image processing instruction sent by the terminal device, a first face image is input into a preset age conversion model, and a second face image with the face age smaller than a second age threshold value is obtained. Therefore, subsequent editing operation can be carried out on the first face image according to the second face image, and the target image with the face age smaller than the second age threshold value is obtained. And sending the target face image to the terminal equipment, so that the user can process the target image after age conversion on the terminal equipment. Be different from the operation that directly increases the sticker to the image that image acquisition device gathered among the prior art, adjust through the age to people's face in the first face image to first face image changes great, and is interesting stronger, and then can improve user experience.
Fig. 1 is a schematic diagram of a network architecture based on the present disclosure, and as shown in fig. 1, the network architecture based on the present disclosure at least includes: a terminal device 1 and an image processing apparatus 2. Wherein, the image processing device 2 is written by C/C + +, java, shell or Python; the terminal device 1 may be a desktop computer, a tablet computer, or the like. The terminal device 1 is in communication connection with the image processing apparatus 2, so that information interaction can be performed between the terminal device and the image processing apparatus.
Fig. 2 is a schematic flowchart of an image processing method according to a first embodiment of the disclosure, and as shown in fig. 2, the method includes:
The execution subject of the present embodiment is an image processing apparatus. The image processing device is in communication connection with the data server, so that information interaction can be carried out with the data server. The image processing apparatus may be installed in the terminal device, or may be an apparatus independent of the terminal device.
In this embodiment, in order to implement an age conversion operation on a face image, the image processing apparatus may acquire an image processing instruction transmitted by the terminal device, where the image processing instruction specifically includes a first face image to be processed. The age of the face in the first face image is larger than a preset first age threshold. The first age threshold may specifically be 18 years old, i.e. the first face image may be a face image of an adult.
It should be noted that the image processing instruction is generated by clicking an image processing icon preset on the display interface of the terminal device by the user. Fig. 3 is a schematic view of a display interface provided in the embodiment of the present disclosure, and as shown in fig. 3, a user may trigger the image processing icon to generate a corresponding image processing instruction. The user may trigger the image processing icon in any one of a single click, a double click, a long press, a drag, and the like, which is not limited in this disclosure.
In this embodiment, after the image processing instruction is acquired, the first face image may be input into a preset age conversion model according to the image processing instruction, so as to obtain the second face image. And the age of the face in the second face image is smaller than a preset second age threshold value. And a preset difference value exists between the first age threshold and the second age threshold. The second age threshold may specifically be 7 years old, that is, the second facial image is a facial image of a child. For practical applications, for example, after the face image of an adult is input into the face transformation model, the face image of a child corresponding to the adult can be obtained.
It should be noted that the age conversion model may specifically be a neural network model, such as a convolutional neural network model, an antagonistic neural network model, and the like, which is not limited by the present disclosure.
In addition, according to the failure of the training data, the age conversion model can convert an image of an adult into an image of a child's face, and can also convert the image of the child's face into an image of the adult's face. The operation of changing the ages of different ages and different changing orientations is realized.
And 103, editing the first face image according to the second face image to obtain a target image.
In this embodiment, after the second face image is obtained, the first face image may be edited according to the second face image to obtain the target image.
Specifically, on the basis of the first embodiment, the step 103 specifically includes:
determining a set of deformation of the second facial image relative to the first facial image;
and editing the first face image according to the deformation array and a target mask corresponding to a preset average face image to obtain the target image.
In this embodiment, in order to implement an age conversion operation on a face region, a deformation array of the second face image relative to the first face image may be first determined, wherein the deformation array specifically characterizes the deformation size of each pixel on the X/Y axis. And editing the first face image according to the deformation array and a target mask corresponding to a preset average face image to obtain the target image. The average face is obtained by summarizing a preset large number of face images, and has the face features of most people.
Specifically, on the basis of any of the above embodiments, the value corresponding to the smooth mask is between 0 and 1, and the value of the smooth mask is proportional to the probability that the pixel belongs to the face region;
correspondingly, the editing operation of the first face image according to the deformation array and the target mask corresponding to the preset average face image includes:
and multiplying the deformation array by the numerical value of the target mask to obtain the target image.
In this embodiment, the value corresponding to the smooth mask is specifically between 0 and 1, the more the value tends to 1, the greater the probability that the pixel point belongs to the face region is represented, and the more the value tends to 0, the greater the probability that the pixel point belongs to the non-face region is represented. Based on the above numerical values, the deformation array can be multiplied by the numerical value of the target mask, so as to realize accurate age conversion operation on the face area and obtain the target image after the age conversion. For example, in practical applications, in the target mask, the value of the face region may be specifically 1, and the value of the non-face region may be specifically 0, and the value of the face region deformation array is not changed and the value of the non-face region deformation array is 0 by multiplying the deformation array by the value of the target mask, so that the optimization operation on only the face region can be realized.
And 104, sending the target image to the terminal equipment so that a user can edit the target image.
In this embodiment, after the target image is obtained, the target image may be transmitted to a terminal device of a user for display. Therefore, after the user views the target image through the display interface of the terminal device, whether the target image needs to be subjected to personalized editing processing can be determined according to the requirements of the user.
According to the image processing method provided by the embodiment, the first face image is input into the preset age conversion model according to the image processing instruction sent by the terminal device, and the second face image with the face age smaller than the second age threshold is obtained. Therefore, subsequent editing operation can be carried out on the first face image according to the second face image, and the target image with the face age smaller than the second age threshold value is obtained. And sending the target face image to the terminal equipment, so that the user can process the age-converted target image on the terminal equipment. Be different from the operation that directly increases the sticker to the image that image acquisition device gathered among the prior art, adjust through the age to people's face in the first face image to first face image changes great, and is interesting stronger, and then can improve user experience.
In addition, in order to improve the accuracy of image processing, the face region in the first face image needs to be accurately cropped, so that the first face image is obtained by cropping according to a preset average face image. The human face image collected by the image collecting device is cut according to the average face image to obtain the first human face image, so that the region of the human face can be accurately obtained.
Fig. 4 is a schematic flowchart of an image processing method according to a second embodiment of the present disclosure, where on the basis of the first embodiment, the method further includes:
In the present embodiment, in order to enable acquisition of the first face image from the average face image, it is first necessary to obtain the average face image. Specifically, a preset first face set may be obtained from the data server, where the first face set includes a plurality of third face images, and each of the third face images has a consistent size and includes at least one key point. Fig. 5 is a further system structure diagram based on the present disclosure, and as shown in fig. 5, the system structure based on the present disclosure further includes a data server 3, and the data server 3 may be a cloud server or a server cluster, in which a large amount of data is stored. The image processing apparatus 2 is connected to the terminal device 1 and the data server 3 in a communication manner, and can exchange information with the terminal device 1 and the data server 3, respectively. And respectively determining the coordinate information of each key point in each third face image aiming at each third face image, and further calculating the average coordinate of each key point of a plurality of third face images in the first face set to obtain the average coordinate of at least one key point.
Further, image deformation may be performed on a third face image in the third face image set according to the average coordinate of at least one key point, so that a distance between the coordinate of the key point extracted for each object in the third face image and the corresponding average coordinate is less than or equal to a preset distance, and a deformed face image set is obtained. It can be understood that after the third face image is deformed, the deformed third face image is obtained.
Specifically, for the third facial image in the third facial image set: firstly, the execution main body can deform the third face image according to a transformation matrix input by a user to obtain a deformed third face image; then, whether the distance between the coordinate of the key point extracted aiming at each object and the corresponding average coordinate in the deformed third face image is smaller than or equal to the preset distance or not can be determined; if the distance is smaller than or equal to the preset distance, obtaining a third face image after deformation; if the distance is greater than the preset distance, the third face image can be deformed according to the transformation matrix input by the user again until the distance between the coordinate of the key point extracted aiming at each object and the corresponding average coordinate in the deformed third face image is less than or equal to the preset distance.
Specifically, a Moving Least Square (MLS) method may be used to perform image deformation on the third facial image in the third facial image set. It should be noted that the executing subject may also perform image transformation on the third face image through another image transformation algorithm. It should be noted that the third face image is deformed by the moving least square method, so that the situation that a user inputs a transformation matrix for many times can be avoided, and the time for image deformation of the third face image is shortened.
In this embodiment, after obtaining the deformed face image set, an average face image of the deformed third face image set may be further determined. And the pixel value of each pixel point in the average face image is the average value of the pixel values of the corresponding pixel points in the deformed third face image set.
Specifically, the image may be essentially regarded as a pixel value matrix formed by pixel values of the pixels, that is, each deformed third face image may be regarded as a pixel value matrix. In this way, the image processing apparatus can average the matrix of pixel values corresponding to the set of deformed face images. It is understood that the pixel value matrix obtained after averaging is the above average face image. The above-described averaging of the pixel value matrices may be averaging of pixel values at the same position in the pixel value matrix.
Further, a target number of second face images with key points extracted can be selected from the second face image set. The target number may be preset or determined according to actual requirements. In order to facilitate subsequent operations, each second face image can be cut according to a preset target size, and a third face image set is obtained. The target size may be obtained according to an empirical value, or may be set by the user, which is not limited by the present disclosure.
In the image processing method provided in this embodiment, the average coordinates of the key points indicating the same object are determined by referring to the coordinates of the key points extracted from each third face image in the third face image set, and on the basis of obtaining the average coordinates, the third face image is subjected to image deformation to obtain a deformed third face image set, so as to generate an average face image of the deformed third face image set. In the process of generating the average face image, triangulation on each third face image is not needed, so that the calculation amount of a server is reduced, and a foundation is provided for adjusting the age of the subsequent first face image.
Fig. 6 is a schematic flowchart of an image processing method according to a third embodiment of the present disclosure, where on the basis of any of the foregoing embodiments, before step 102, the method further includes:
301, acquiring a data set to be trained from a data server, wherein the data set to be trained comprises a plurality of first face images and second face images corresponding to the first face images;
and 303, inputting the generated image and the second face image into the discriminator to enable the discriminator to judge the authenticity of the generated image and the second face image, and performing supervision training on the generator according to a judgment result until the model converges to obtain the age conversion model.
In the embodiment, the age conversion model comprises a generator and a discriminator, and the generator and the discriminator are in mutual confrontation in the training process, so that the image generated by the generator can be promoted to be closer to a real image. Fig. 7 is a network architecture diagram of an age transformation model provided in an embodiment of the present disclosure, and as shown in fig. 7, the age transformation model includes a generator and a discriminator, the generator is configured to generate a virtual image according to an image to be trained, and the discriminator is configured to discriminate the virtual image output by the discriminator and an input second face image, and urge the generator to generate a more real image. Specifically, in order to implement training of the age conversion model, a preset data set to be trained needs to be acquired from a data server, where the data set to be trained includes a plurality of first face images and a second face image corresponding to the first face images. And the data set to be trained is an open source data set. And inputting each first face image in the data set to be trained into the generator to obtain a generated image corresponding to the first face image output by the generator.
And inputting the generated image output by the generator and the second face image into the discriminator together. After the generated image is received by the discriminator, the generated image is discriminated as false as possible, when the second face image is received, the generated image is discriminated as true as possible and continuously confronts the generator, and the truth degree of the generated image output by the supervision generator is higher and higher. The generator and the arbiter continuously compete until the model converges. So that the generator can be used to perform the operation of generating the second face image.
In the image processing method provided by the embodiment, the preset antagonistic neural network is trained on the image set to be trained in advance, and the generator can be supervised by the discriminator, so that the second face image generated by the generator is higher in true degree, and the target image obtained after the second face image is edited and processed correspondingly is more interesting.
Fig. 8 is a schematic flowchart of an image processing method according to a fourth embodiment of the present disclosure, and on the basis of any of the foregoing embodiments, as shown in fig. 8, after step 104, the method further includes:
and 402, editing the target image according to the target special effect information.
In the present embodiment, after the age-converted target image is obtained, the target image may be transmitted to the terminal device. Accordingly, after the terminal device acquires the target image, the terminal device can display the target image on the display interface. The user can edit the target image on the terminal equipment according to the self requirement. Specifically, the user may generate an editing instruction on the terminal device, where the editing instruction includes the target special effect information. The target special effect information includes at least one of a face adjustment special effect, a hair style adjustment special effect, and a sticker special effect. After the editing instruction is acquired, the target image can be edited according to the target special effect information.
With practical application distance, the user can carry out 'face thinning' operation to the face in the target image, because this target image is obtained after carrying out age conversion to the first face image, the difference between it and the first face image is great, and it is better to correspondingly carry out the editing operation effect according to this target image.
According to the image processing method provided by the embodiment, the editing instruction sent by the terminal device is obtained, and the target image is edited according to the target special effect information in the editing instruction. Thereby improving the effect of face editing processing.
Fig. 9 is a schematic structural diagram of an image processing apparatus according to a fifth embodiment of the present disclosure, and as shown in fig. 9, the apparatus includes: the image processing method comprises an acquisition module 51, a conversion module 52, an editing module 53 and a sending module 54, wherein the acquisition module 51 is used for acquiring an image processing instruction sent by a terminal device, the image processing instruction comprises a first face image to be processed, and the age of a face in the face image is greater than a preset first age threshold; a conversion module 52, configured to input the face image into a preset age conversion model according to the image processing instruction, and obtain a second face image, where an age of a face in the second face image is smaller than a preset second age threshold, and a preset age difference exists between the first age threshold and the second age threshold; the editing module 53 is configured to perform an editing operation on the first face image according to the second face image to obtain a target image; a sending module 54, configured to send the target image to the terminal device, so that a user performs editing processing on the target image.
Further, on the basis of the fifth embodiment, the editing module is configured to:
determining a deformation number group of the second face image relative to the first face image;
and editing the first face image according to the deformation array and a target mask corresponding to a preset average face image to obtain the target image.
Further, on the basis of the fifth embodiment, the value corresponding to the target mask is between 0 and 1, and the value of the target mask is in direct proportion to the probability that the pixel point belongs to the face region;
accordingly, the editing module is to:
and multiplying the deformation array by the numerical value of the target mask to obtain the target image.
Further, on the basis of the fifth embodiment, the image processing instruction is generated by a user clicking an image processing icon preset on the display interface of the terminal device.
The image processing apparatus provided in this embodiment inputs the first face image into the preset age conversion model according to the image processing instruction sent by the terminal device, and obtains the second face image with the face age smaller than the second age threshold. Therefore, subsequent editing operation can be carried out on the first face image according to the second face image, and the target image with the face age smaller than the second age threshold value is obtained. And sending the target face image to the terminal equipment, so that the user can process the target image after age conversion on the terminal equipment. Be different from the operation that directly increases the sticker to the image that image acquisition device gathered among the prior art, adjust through the age to people's face in the first face image to first face image changes great, and is interesting stronger, and then can improve user experience.
Further, in the fifth embodiment, the first face image is obtained by cropping according to a preset average face image.
Fig. 10 is a schematic structural diagram of an image processing apparatus according to a sixth embodiment of the present disclosure, and on the basis of the fifth embodiment, the apparatus further includes: the face image processing device comprises a face set acquisition module 61, a coordinate determination module 62, a calculation module 63, a deformation module 64 and an average face determination module 65, wherein the face set acquisition module 61 is used for acquiring a preset third face set, and third face images in the third face set have consistent sizes and all include at least one key point; a coordinate determining module 62, configured to determine at least one key point coordinate corresponding to each third face image in the third face set; a calculating module 63, configured to calculate, according to at least one key point coordinate corresponding to each third face image, an average coordinate of each key point corresponding to the third face set; a deformation module 64, configured to perform a deformation operation on each third face image according to the average coordinate of each key point to obtain a deformed third face image, where a distance between each key point coordinate in the deformed third face image and the average coordinate is smaller than a preset threshold; an average face determining module 65, configured to determine an average face image corresponding to the deformed third face image set, where a pixel value of each pixel in the average face image is an average value of pixel values of corresponding pixels in each deformed third face image in the deformed third face image set.
The image processing apparatus provided in this embodiment determines, by referring to the coordinates of the extracted key points of each third face image in the third face image set, average coordinates of key points indicating the same object, and obtains a deformed third face image set by performing image deformation on the third face image on the basis of the obtained average coordinates, thereby generating an average face image of the deformed third face image set. In the process of generating the average face image, triangulation on each third face image is not needed, so that the calculation amount of a server is reduced, and a foundation is provided for adjusting the age of the subsequent first face image.
Fig. 11 is a schematic structural diagram of an image processing apparatus according to a seventh embodiment of the present disclosure, in which on the basis of any of the foregoing embodiments, the age conversion model includes a generator and an arbiter; correspondingly, the device further comprises: the system comprises a data set acquisition module 71, an input module 72 and a training module 73, wherein the data set acquisition module 71 is configured to acquire a data set to be trained from a data server, and the data set to be trained includes a plurality of first face images and second face images corresponding to the first face images; an input module 72, configured to input the first face image into a generator, and obtain a generated image corresponding to the first face image; and the training module 73 is configured to input the generated image and the second face image into the discriminator, so that the discriminator determines the authenticity of the generated image and the second face image, and performs supervised training on the generator according to a determination result until the model converges, so as to obtain the age conversion model.
The image processing apparatus provided by this embodiment trains the preset antagonistic neural network through the pre-trained image set, and can supervise the generator through the discriminator, so that the second face image generated by the generator is more realistic, and accordingly, the target image obtained after editing processing according to the second face image is more interesting.
Fig. 12 is a schematic structural diagram of an image processing apparatus according to an eighth embodiment of the present disclosure, and on the basis of any of the foregoing embodiments, as shown in fig. 12, the apparatus further includes: the image editing system comprises an instruction acquisition module 81 and an editing module 82, wherein the instruction acquisition module 81 is used for acquiring an image editing instruction sent by terminal equipment, and the image editing instruction comprises target special effect information; and the editing module 82 is configured to perform an editing operation on the target image according to the target special effect information.
Further, on the basis of any of the above embodiments, the target effect information includes at least one of a face adjustment effect, a hair style adjustment effect, and a sticker effect.
The image processing apparatus provided in this embodiment, by acquiring an editing instruction sent by a terminal device, performs an editing operation on a target image according to target special effect information in the editing instruction. Thereby improving the effect of face editing processing.
Fig. 13 is a schematic structural diagram of an image processing apparatus according to a ninth embodiment of the present disclosure, and as shown in fig. 13, the image processing apparatus includes: a memory 91, a processor 92;
a memory 91; a memory 92 for storing instructions executable by the processor 92;
wherein the processor 92 is configured to execute the image processing method according to any of the above embodiments by the processor 92.
The memory 91 stores programs. In particular, the program may include program code comprising computer operating instructions. The memory 91 may comprise a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.
The processor 92 may be a Central Processing Unit (CPU), an Application Specific Integrated Circuit (ASIC), or one or more Integrated circuits configured to implement the embodiments of the present disclosure.
Alternatively, in a specific implementation, if the memory 91 and the processor 92 are implemented independently, the memory 91 and the processor 92 may be connected to each other through a bus and perform communication with each other. The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, an Extended ISA (EISA) bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown in FIG. 13, but this is not intended to represent only one bus or type of bus.
Alternatively, in a specific implementation, if the memory 91 and the processor 92 are integrated on a chip, the memory 91 and the processor 92 may complete the same communication through an internal interface.
Yet another embodiment of the present disclosure further provides a computer-readable storage medium, in which computer-executable instructions are stored, and when the computer-executable instructions are executed by a processor, the computer-readable storage medium is used for implementing the image processing method according to any one of the above embodiments.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working process of the apparatus described above may refer to the corresponding process in the foregoing method embodiment, and is not described herein again.
Those of ordinary skill in the art will understand that: all or a portion of the steps of implementing the above-described method embodiments may be performed by hardware associated with program instructions. The program may be stored in a computer-readable storage medium. When executed, the program performs steps comprising the method embodiments described above; and the aforementioned storage medium includes: various media that can store program codes, such as ROM, RAM, magnetic or optical disks.
Finally, it should be noted that: the above embodiments are only used for illustrating the technical solutions of the present disclosure, and not for limiting the same; while the present disclosure has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art will understand that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present disclosure.
Claims (18)
1. An image processing method, comprising:
acquiring an image processing instruction sent by terminal equipment, wherein the image processing instruction comprises a first face image to be processed, and the age of a face in the face image is greater than a preset first age threshold;
inputting the face image into a preset age conversion model according to the image processing instruction to obtain a second face image, wherein the age of the face in the second face image is smaller than a preset second age threshold, and a preset age difference value exists between the first age threshold and the second age threshold;
editing the first face image according to the second face image to obtain a target image;
sending the target image to the terminal equipment so that a user can edit the target image;
the editing operation of the first human face image according to the second human face image to obtain a target image comprises the following steps:
determining a deformation number group of the second face image relative to the first face image;
and editing the first face image according to the deformation array and a target mask corresponding to a preset average face image to obtain the target image.
2. The method according to claim 1, wherein the value of the target mask is between 0 and 1, and the value of the target mask is proportional to the probability that the pixel belongs to the face region;
correspondingly, the editing operation of the first face image according to the deformation array and the target mask corresponding to the preset average face image includes:
and multiplying the deformation array by the numerical value of the target mask to obtain the target image.
3. The method according to claim 1, wherein the first face image is obtained by cropping according to a preset average face image.
4. The method according to claim 1 or 3, characterized in that the method further comprises:
acquiring a preset third face set, wherein the sizes of third face images in the third face set are consistent and the third face images in the third face set comprise at least one key point;
determining at least one key point coordinate corresponding to each third face image in the third face set;
calculating the average coordinate of each key point corresponding to the third face set according to at least one key point coordinate corresponding to each third face image;
performing deformation operation on each third face image according to the average coordinate of each key point to obtain a deformed third face image, wherein the distance between each key point coordinate in the deformed third face image and the average coordinate is smaller than a preset threshold value;
and determining an average face image corresponding to the deformed third face image set, wherein the pixel value of each pixel point in the average face image is the average value of the pixel values of corresponding pixel points in the deformed third face image set.
5. The method according to any one of claims 1-3, wherein the age conversion model comprises a generator and an arbiter;
correspondingly, before the inputting the face image to a preset age conversion model according to the image processing instruction, the method further includes:
acquiring a data set to be trained from a data server, wherein the data set to be trained comprises a plurality of first face images and second face images corresponding to the first face images;
inputting the first face image into a generator to obtain a generated image corresponding to the first face image;
and inputting the generated image and the second face image into the discriminator to enable the discriminator to judge the authenticity of the generated image and the second face image, and performing supervision training on the generator according to a judgment result until the model converges to obtain the age conversion model.
6. The method according to any one of claims 1 to 3, wherein after the sending the target image to the terminal device, further comprising:
acquiring an image editing instruction sent by terminal equipment, wherein the image editing instruction comprises target special effect information;
and editing the target image according to the target special effect information.
7. The method of claim 6, wherein the target effect information includes at least one of a face adjustment effect, a hair style adjustment effect, and a sticker effect.
8. The method according to any one of claims 1-3, wherein the image processing instruction is generated by a user clicking a preset image processing icon on the display interface of the terminal device.
9. An image processing apparatus characterized by comprising:
the terminal equipment comprises an acquisition module, a processing module and a processing module, wherein the acquisition module is used for acquiring an image processing instruction sent by the terminal equipment, the image processing instruction comprises a first face image to be processed, and the age of a face in the face image is greater than a preset first age threshold value;
the conversion module is used for inputting the face image into a preset age conversion model according to the image processing instruction to obtain a second face image, wherein the age of the face in the second face image is smaller than a preset second age threshold, and a preset age difference value exists between the first age threshold and the second age threshold;
the editing module is used for carrying out editing operation on the first face image according to the second face image to obtain a target image;
the sending module is used for sending the target image to the terminal equipment so as to enable a user to edit the target image;
the editing module is used for:
determining a set of deformation of the second facial image relative to the first facial image;
and editing the first face image according to the deformation array and a target mask corresponding to a preset average face image to obtain the target image.
10. The apparatus according to claim 9, wherein the value of the target mask is between 0 and 1, and the value of the target mask is proportional to the probability that the pixel belongs to the face region;
accordingly, the editing module is to:
and multiplying the deformation array by the numerical value of the target mask to obtain the target image.
11. The apparatus according to claim 9, wherein the first face image is obtained by cropping according to a preset average face image.
12. The apparatus of claim 9 or 11, further comprising:
the face set acquisition module is used for acquiring a preset third face set, wherein the third face images in the third face set are consistent in size and all comprise at least one key point;
the coordinate determination module is used for determining at least one key point coordinate corresponding to each third face image in the third face set;
the calculating module is used for calculating the average coordinate of each key point corresponding to the third face set according to the at least one key point coordinate corresponding to each third face image;
the deformation module is used for carrying out deformation operation on each third face image according to the average coordinate of each key point to obtain a deformed third face image, wherein the distance between each key point coordinate in the deformed third face image and the average coordinate is smaller than a preset threshold value;
and the average face determining module is used for determining an average face image corresponding to the deformed third face image set, wherein the pixel value of each pixel point in the average face image is the average value of the pixel values of corresponding pixel points in the deformed third face image set.
13. The apparatus according to any one of claims 9-11, wherein the age conversion model comprises a generator and a discriminator;
correspondingly, the device further comprises:
the data set acquisition module is used for acquiring a data set to be trained from a data server, wherein the data set to be trained comprises a plurality of first face images and second face images corresponding to the first face images;
the input module is used for inputting the first face image into a generator to obtain a generated image corresponding to the first face image;
and the training module is used for inputting the generated image and the second face image into the discriminator so as to enable the discriminator to judge the authenticity of the generated image and the second face image, and performing supervision training on the generator according to a judgment result until the model converges to obtain the age conversion model.
14. The apparatus according to any one of claims 9-11, further comprising:
the instruction acquisition module is used for acquiring an image editing instruction sent by terminal equipment, wherein the image editing instruction comprises target special effect information;
and the editing module is used for editing the target image according to the target special effect information.
15. The apparatus of claim 14, wherein the target effect information comprises at least one of a face adjustment effect, a hair style adjustment effect, and a sticker effect.
16. The apparatus according to any one of claims 9-11, wherein the image processing instruction is generated by a user clicking a preset image processing icon on the display interface of the terminal device.
17. An image processing apparatus characterized by comprising: a memory, a processor;
a memory; a memory for storing the processor-executable instructions;
wherein the processor is configured to perform the image processing method of any one of claims 1-8 by the processor.
18. A computer-readable storage medium having computer-executable instructions stored therein, which when executed by a processor, are configured to implement the image processing method of any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010423985.4A CN111461971B (en) | 2020-05-19 | 2020-05-19 | Image processing method, device, equipment and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010423985.4A CN111461971B (en) | 2020-05-19 | 2020-05-19 | Image processing method, device, equipment and computer readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111461971A CN111461971A (en) | 2020-07-28 |
| CN111461971B true CN111461971B (en) | 2023-04-18 |
Family
ID=71684765
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010423985.4A Active CN111461971B (en) | 2020-05-19 | 2020-05-19 | Image processing method, device, equipment and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111461971B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115410262B (en) * | 2022-10-09 | 2023-06-23 | 上海本趣网络科技有限公司 | Face image information prediction system |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108416310A (en) * | 2018-03-14 | 2018-08-17 | 百度在线网络技术（北京）有限公司 | Method and apparatus for generating information |
| CN110288513A (en) * | 2019-05-24 | 2019-09-27 | 北京百度网讯科技有限公司 | For changing the method, apparatus, equipment and storage medium of face character |
| CN111079588A (en) * | 2019-12-03 | 2020-04-28 | 北京字节跳动网络技术有限公司 | Image processing method, device and storage medium |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR101635730B1 (en) * | 2014-10-08 | 2016-07-20 | 한국과학기술연구원 | Apparatus and method for generating montage, recording medium for performing the method |
| US10621771B2 (en) * | 2017-03-21 | 2020-04-14 | The Procter & Gamble Company | Methods for age appearance simulation |
-
2020
- 2020-05-19 CN CN202010423985.4A patent/CN111461971B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108416310A (en) * | 2018-03-14 | 2018-08-17 | 百度在线网络技术（北京）有限公司 | Method and apparatus for generating information |
| CN110288513A (en) * | 2019-05-24 | 2019-09-27 | 北京百度网讯科技有限公司 | For changing the method, apparatus, equipment and storage medium of face character |
| CN111079588A (en) * | 2019-12-03 | 2020-04-28 | 北京字节跳动网络技术有限公司 | Image processing method, device and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111461971A (en) | 2020-07-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10832039B2 (en) | Facial expression detection method, device and system, facial expression driving method, device and system, and storage medium | |
| WO2022156640A1 (en) | Gaze correction method and apparatus for image, electronic device, computer-readable storage medium, and computer program product | |
| CN109934065B (en) | Method and device for gesture recognition | |
| CN109815881A (en) | Training method, the Activity recognition method, device and equipment of Activity recognition model | |
| CN108921782A (en) | A kind of image processing method, device and storage medium | |
| CN112232258B (en) | Information processing method, device and computer readable storage medium | |
| CN111652974B (en) | Method, device, equipment and storage medium for constructing three-dimensional face model | |
| CN111008935B (en) | Face image enhancement method, device, system and storage medium | |
| CN110781770B (en) | Living body detection method, device and equipment based on face recognition | |
| CN114746898A (en) | Method and system for generating trisection images of image matting | |
| WO2023151237A1 (en) | Face pose estimation method and apparatus, electronic device, and storage medium | |
| WO2023173646A1 (en) | Expression recognition method and apparatus | |
| CN111798551A (en) | Virtual expression generation method and device | |
| CN112561879B (en) | Ambiguity evaluation model training method, image ambiguity evaluation method and image ambiguity evaluation device | |
| CN111369428A (en) | Virtual head portrait generation method and device | |
| CN111583280B (en) | Image processing method, device, equipment and computer readable storage medium | |
| CN112492297B (en) | Video processing method and related equipment | |
| CN110674837A (en) | Video similarity obtaining method and device, computer equipment and storage medium | |
| WO2024012333A1 (en) | Pose estimation method and apparatus, related model training method and apparatus, electronic device, computer readable medium and computer program product | |
| CN112257729B (en) | Image recognition method, device, equipment and storage medium | |
| CN112381071A (en) | Behavior analysis method of target in video stream, terminal device and medium | |
| CN113160231A (en) | Sample generation method, sample generation device and electronic equipment | |
| CN111667504A (en) | Face tracking method, device and equipment | |
| CN111461971B (en) | Image processing method, device, equipment and computer readable storage medium | |
| KR20160046399A (en) | Method and Apparatus for Generation Texture Map, and Database Generation Method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |