WO2022089185A1

WO2022089185A1 - Image processing method and image processing device

Info

Publication number: WO2022089185A1
Application number: PCT/CN2021/123080
Authority: WO
Inventors: 赵明菲; 闻兴
Original assignee: 北京达佳互联信息技术有限公司
Priority date: 2020-10-30
Filing date: 2021-10-11
Publication date: 2022-05-05
Also published as: CN113012031A

Abstract

The present invention relates to an image processing method and an image processing device. The image processing method comprises: recognizing a face region in a first image; performing face thinning on the face region in the first image on the basis of a face thinning algorithm so as to obtain a second image after the face thinning; and repairing a deformed region generated due to the face thinning in the second image so as to obtain a repaired third image.

Description

Image processing method and image processing device

CROSS-REFERENCE TO RELATED APPLICATIONS

This disclosure claims the priority of a Chinese patent application with application number 202011192273.2 and titled "Image Processing Method and Image Processing Device" filed with the China Patent Office on October 30, 2020, the entire contents of which are incorporated in this disclosure by reference .

technical field

The present disclosure relates to the technical field of image processing, and more particularly, to an image processing method and an image processing apparatus.

Background technique

In application scenarios such as live video, short video, and photography, a very commonly used technology is face-lifting, which can make the user's face smaller and achieve better subjective effects.

SUMMARY OF THE INVENTION

The present disclosure provides an image processing method and an image processing apparatus.

According to a first aspect of the embodiments of the present disclosure, there is provided an image processing method, comprising: identifying a face region in a first image; performing face-lifting on the face region in the first image based on a face-lifting algorithm, to obtain a face-lifted face the second image; perform repairing on the deformed area in the second image resulting from performing face-lifting to obtain a repaired third image.

In some embodiments, the deformed area may be an area other than the face area after face reduction in a predetermined area including the face area involved in performing face reduction in the first image.

In some embodiments, the performing inpainting on the deformed region in the second image resulting from performing face reduction may include: performing inpainting by filling the deformed region with background pixels.

In some embodiments, the performing inpainting by filling the deformed area with background pixels may include: based on the deformed area and the first image or first image, using an image inpainting algorithm to fill the deformed area background pixels.

In some embodiments, the performing inpainting by filling the deformed area with background pixels may include: filling the deformed area with background pixels based on the background image and the second image, wherein the background image is a The first image has a pure background image of the same scene.

In some embodiments, the filling the deformed area with background pixels based on the background image and the second image may include: searching the background image for the same area as the deformed area; using the search in the background image The pixel values of the obtained pixels in the region replace the pixel values of the pixels in the deformed region.

According to a second aspect of the embodiments of the present disclosure, there is provided an image processing apparatus, comprising: an identification unit configured to identify a face region in a first image; a face reduction unit configured to performing face-lifting on the face region to obtain a second image after face-lifting; the repairing unit is configured to perform repairing on a deformed region in the second image that is generated by performing face-lifting, so as to generate a repaired third image.

In some embodiments, the inpainting unit may be configured to perform inpainting by filling the deformed region with background pixels.

In some embodiments, the inpainting unit may be configured to fill the deformed region with background pixels using an image inpainting algorithm based on the deformed region and the first image or the first image.

In some embodiments, the repair unit may be configured to fill the deformed region with background pixels based on a background image and a second image, wherein the background image is a pure background image having the same scene as the first image.

In some embodiments, the repairing unit may be configured to: search for an area in the background image that is the same as the deformed area; replace the deformed area with pixel values of pixels of the searched area in the background image The pixel value of the pixel in .

According to a third aspect of embodiments of the present disclosure, there is provided an electronic device, comprising: at least one processor; at least one memory storing computer-executable instructions, wherein the computer-executable instructions are executed by the at least one processor when the at least one processor is caused to execute the image processing method according to the present disclosure.

According to a fourth aspect of the embodiments of the present disclosure, there is provided a computer-readable storage medium storing instructions, wherein, when the instructions are executed by at least one processor, the at least one processor is caused to execute the method according to the present disclosure. image processing method.

According to a fifth aspect of the embodiments of the present disclosure, there is provided a computer program product, wherein instructions in the computer program product can be executed by a processor of a computer device to complete the image processing method according to the present disclosure.

According to the image processing method and the image processing apparatus of the present disclosure, a more natural and more realistic face-lifting effect can be obtained by performing restoration on a region that is distorted and deformed by performing face-lifting.

It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure.

Description of drawings

The accompanying drawings, which are incorporated into and constitute a part of this specification, illustrate embodiments consistent with the present disclosure, and together with the description, serve to explain the principles of the present disclosure and do not unduly limit the present disclosure.

FIG. 1 is a schematic diagram illustrating a face-lifting algorithm.

FIG. 2 is a schematic diagram illustrating an implementation scenario of an image processing method and an image processing apparatus according to an exemplary embodiment of the present disclosure.

FIG. 3 is a flowchart illustrating an image processing method according to an exemplary embodiment of the present disclosure.

FIG. 4 is a schematic diagram showing a deformed area due to face-lifting.

FIG. 5 is a schematic diagram illustrating a background frame replacement method according to an exemplary embodiment of the present disclosure.

FIG. 6 is a block diagram illustrating an image processing apparatus according to an exemplary embodiment of the present disclosure.

FIG. 7 is a block diagram of an electronic device according to an exemplary embodiment of the present disclosure.

Detailed ways

In order to make those skilled in the art better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings.

It should be noted that the terms "first", "second" and the like in the description and claims of the present disclosure and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific sequence or sequence. It is to be understood that the data so used may be interchanged under appropriate circumstances such that the embodiments of the disclosure described herein can be practiced in sequences other than those illustrated or described herein. The implementations described in the following examples are not intended to represent all implementations consistent with this disclosure. Rather, they are merely examples of apparatus and methods consistent with some aspects of the present disclosure as recited in the appended claims.

It should be noted here that "at least one of several items" in the present disclosure all means including "any one of the several items", "a combination of any of the several items", The three categories of "the whole of the several items" are juxtaposed. For example, "including at least one of A and B" includes the following three parallel situations: (1) including A; (2) including B; (3) including A and B. Another example is "execute at least one of step 1 and step 2", which means the following three parallel situations: (1) execute step 1; (2) execute step 2; (3) execute step 1 and step 2.

The existing face-lifting algorithms are mainly based on the detection of key points of the face, then densify the key points of the face, and finally use the triangulation method to achieve the effect of face-lifting. FIG. 1 is a schematic diagram illustrating a face-lifting algorithm. As shown in Figure 1, the process of the face-lifting algorithm may include: (1) As shown in (a) in Figure 1, first, obtain basic information on key points of the face, mainly including eyebrows, eyes, nose, mouth, face The 106 key points of the outer contour; (2) As shown in (b) in Figure 1, secondly, based on the detected 106 key points, the facial key points are densified, and additional key points are inserted, such as The forehead area and the peripheral area of the face are limited so that it can cover the entire face area; (3) As shown in (c) in Figure 1, finally, based on the face key points after densification, the entire face is constructed for it The triangulation of the entire face area (Delaunay Triangulation) is realized. The triangulation divides the face into multiple non-overlapping triangular areas, and then the area transformation can achieve the effect of thin face. In the process of region transformation, by translating the vertices of the triangulation, then updating the translated vertices to the corresponding texture coordinates, and rendering through openGL or D3D, so as to realize the deformation of the entire associated triangulation. Although this achieves the face-reduction effect, it will also lead to deformation or unnaturalness of the non-face area (as shown in (c) in Figure 1) involved in the triangular mesh of the face-reduction algorithm. The non-face area includes the background area around the original face area and the area in the original face area that is thinned by the face thinning algorithm. In order to solve the above problems, the present disclosure proposes an image processing method and an image processing apparatus, which can perform restoration on an area deformed by a face-reduction operation after a face-reduction operation is performed on an image, so as to obtain a more natural face-reduction image. Hereinafter, the image processing method and the image processing apparatus according to the present disclosure will be described in detail with reference to FIGS. 2 to 7 .

2, in the network live broadcast system, the host can use the live broadcast equipment 201 to shoot the live broadcast program and upload the live broadcast program to the server 202 in the live broadcast room through the client of the live broadcast equipment 201, and the server 202 distributes the live broadcast program to the host who enters the host. The client terminal of the

user terminal

203 or 204 in the live broadcast room can present the live broadcast program to the users watching the live broadcast. Here, the live broadcast device 201 may be any device including a shooting function or a device capable of being connected with the shooting device, for example, a mobile phone, a portable computer, a tablet computer, a video camera, and the like. In order to beautify the image of the host seen by the users watching the live broadcast, the client used by the host during the live broadcast can perform face-lifting on the part of the face in the video and/or image captured by the live-streaming device 201, and include the face-lifting performed The resulting live video and/or image live program is uploaded to the server 202 and distributed to each

user terminal

203 or 204 for viewing by the user, so the video and/or image of the anchor with a face-lifted and more beautified image can be viewed. Therefore, the image processing method and the image processing apparatus according to the present disclosure can be applied to this live broadcast scene.

In addition, the image processing method and image processing apparatus according to the present disclosure can be applied to any scene where face reduction can be performed, such as a short video recording scene, a photographing scene, a self-portraiting scene, and the like, in addition to a live broadcast scene.

Referring to FIG. 3, in step 301, a face region in a first image may be identified. Here, the first image may be an image captured by photographing, or may be an image of a video obtained by capturing a video. In addition, the first image may be obtained in real-time from the photographing device, or may be obtained from local storage or a local database as needed, or received from an external data source (eg, the Internet, a server, a database, etc.) through an input device or transmission medium, the present There is no restriction on this disclosure. Furthermore, the face area may be an area occupied by a face in the first image. The face area may be an area including only a face part, or an area including a face part and related parts such as hair and accessories, which is not limited in the present disclosure. In addition, any possible face recognition method can be used to identify the face region in the image, which is not limited in the present disclosure.

In step 302, a face-lift may be performed on a face region in the first image based on a face-lift algorithm to obtain a face-lifted second image. For example, the above-mentioned triangulation method can be used to perform face reduction on the face region, but any other possible face reduction algorithm can also be used to perform face reduction, which is not limited in the present disclosure. For example, the second image may be an intermediate process image and may not be the actual output image.

When using the face-lifting algorithm to perform face-lifting on the face region, it will inevitably lead to different degrees of distortion and deformation of the surrounding regions of the face region. The surrounding area may include a background area around the original face area and an area in the original face area that is thinned by the face thinning algorithm. FIG. 4 is a schematic diagram showing a deformed area due to face-lifting. As shown in FIG. 4 , (a) in FIG. 4 exemplarily shows a first image before performing face-lifting, and (b) in FIG. 4 exemplarily shows a second image after performing face-lifting. As shown in (a) of FIG. 4 , before performing face-lifting on the face region using the face-lifting algorithm, the first image may include a face region 401 and a background region 402 . In addition, the predetermined area 403 to be involved in performing face-reduction on the face area using the face-reduction algorithm may include the face area 401 and a part of the background area 402 . For example, when face-lifting is performed using a triangulation algorithm, the predetermined area 403 may be an area involved in a triangular mesh (as shown in (c) in FIG. 1 ). As shown in (b) of FIG. 4 , after performing face-lifting on the face region using the face-lifting algorithm, the second image may include a face region 401 ′ and a background region 402 ′, wherein the face region 401 ′ is reduced due to the face-lifting algorithm. Decrease, the background area 402' increases because the reduced part of the face area is filled with background pixels. In addition, the area other than the face area 401' after face reduction in the predetermined area 403' will be deformed, that is, it may be referred to as a deformed area. The deformed area may include a background area 404' around the original face area and an area 405' in the original face area that is thinned by a face thinning algorithm.

Referring back to FIG. 3 , therefore, in step 303 , inpainting may be performed on the deformed region in the second image resulting from performing face-lifting to obtain a repaired third image.

According to an exemplary embodiment of the present disclosure, a deformed area resulting from performing face reduction may be first determined. For example, the deformed area may be determined as an area other than the face area after face reduction in a predetermined area including a face area involved in performing face reduction in the first image, for example, in (b) of FIG. 4 The deformed area is an area (404'+405') other than the face area 401' in the predetermined area 403'. For another example, the deformed region can also be determined by comparing the pixel values around the face region of the second image after face-lifting with the pixel values around the face region of the first image before face-lifting. Of course, any possible method can be used to determine the deformation area, which is not limited in the present disclosure.

According to an exemplary embodiment of the present disclosure, inpainting may be performed on the deformed region by filling the deformed region with background pixels. For example, the inpainting can be performed on the deformed regions by an image inpainting algorithm or a method of background frame replacement. Of course, the present disclosure is not limited to these repair methods, and any possible repair method can be used to perform repair on the deformed region. The following describes in detail how to perform inpainting on deformed regions through image inpainting algorithms or background frame replacement.

According to an exemplary embodiment of the present disclosure, the deformed region may be filled with background pixels using an image inpainting algorithm based on the deformed region and the first image or the first image. For example, image inpainting algorithms may include traditional inpainting algorithms (non-deep learning algorithms) and deep learning algorithms. Conventional patching algorithms may include patch-based methods and diffusion-based methods. When using the image patch-based method, an image patch similar to the deformed area can be filled in the deformed area by searching the first image before performing face-lifting. When the diffusion-based method is used, the pixels at the edge of the deformed area can be in-grown according to the properties of the corresponding area of the first image before performing face-lifting, and the entire deformed area can be filled by diffusion. Deep learning algorithms may include convolutional neural network (CNN) based methods, generative adversarial network (GAN) based methods, recurrent neural network (RNN) based methods, and the like. A mask may be generated based on the deformed area, and based on the second image after performing face reduction and the generated mask, image inpainting may be performed on the deformed area using a deep learning algorithm. The specific method may be inputting the second image and the generated mask after performing face-lifting into the model based on the deep learning algorithm, and the model based on the deep learning algorithm outputs the third image after the deformed area has been repaired.

According to an exemplary embodiment of the present disclosure, under the condition that a pure background image with the same scene as the first image can be obtained, a simpler and faster background frame replacement method can be used to repair the deformed region. For example, but not limited to, when the photographing device for photographing the first image is stationary and the position is fixed, the background image before adding the human face and the first image after adding the human face can be obtained respectively. For example, in a video scene, multiple frames of video images can be continuously captured, and the shooting device can first capture image frames with only background to obtain background frame images, and then allow the user to shoot in front of the shooting device to obtain video frame images (for example, the first image).

After the background image is acquired, the deformed area may be filled with background pixels based on the background image and the second image. For example, the background image may be searched for the same area as the deformed area, and the pixel value of the pixel in the deformed area may be replaced with the pixel value of the pixel of the searched area in the background image. FIG. 5 is a schematic diagram illustrating a background frame replacement method according to an exemplary embodiment of the present disclosure. Referring to FIG. 5 , a pure background image 501 having the same scene as the first image can be acquired. The pure background image 501 is searched for the same area 502 (shown by hatching) as the deformed area (404'+405') shown in (b) of Fig. 4 . The pixel values of the pixels in the deformed area (404'+405') are replaced with the pixel values of the pixels in the searched area 502.

6 , an image processing apparatus 600 according to an exemplary embodiment of the present disclosure may include a recognition unit 601 , a face reduction unit 602 , and a repair unit 603 .

The identifying unit 601 can identify the face region in the first image. Here, the first image may be an image captured by photographing, or may be an image of a video obtained by capturing a video. In addition, the first image may be obtained in real-time from the photographing device, or may be obtained from local storage or a local database as needed, or received from an external data source (eg, the Internet, a server, a database, etc.) through an input device or transmission medium, the present There is no restriction on this disclosure. Furthermore, the face area may be an area occupied by a face in the first image. The face area may be an area including only a face part, or an area including a face part and related parts such as hair and accessories, which is not limited in the present disclosure. In addition, the recognition unit 601 may use any possible face recognition method to recognize the face area in the image, which is not limited in the present disclosure.

The face-reduction unit 602 may perform face-reduction on the face region in the first image based on a face-reduction algorithm to obtain a face-reduced second image. For example, the face-reduction unit 602 may use the above-mentioned triangulation method to perform face-reduction on the face region, but may also use any other possible face-reduction algorithms to perform face-reduction, which is not limited in the present disclosure. For example, the second image may be an intermediate process image and may not be the actual output image.

When using the face-lifting algorithm to perform face-lifting on the face region, it will inevitably lead to different degrees of distortion and deformation of the surrounding regions of the face region. The surrounding area may include a background area around the original face area and an area in the original face area that is thinned by the face thinning algorithm. Therefore, the inpainting unit 603 may perform inpainting on the deformed region in the second image resulting from performing the face reduction, to obtain a repaired third image.

According to an exemplary embodiment of the present disclosure, the repairing unit 603 may first determine a deformed area resulting from performing face reduction. For example, the repairing unit 603 may determine the deformed area as an area other than the face area after face reduction in the predetermined area including the face area involved in performing face reduction in the first image, for example, (b in FIG. 4 ) The deformed area in ) is the area (404'+405') in the predetermined area 403' except the face area 401'. For another example, the repairing unit 603 may also determine the deformed region by comparing the pixel values around the face region of the second image after face reduction with the pixel values around the face region of the first image before face reduction. Of course, any possible method can be used to determine the deformation area, which is not limited in the present disclosure.

According to an exemplary embodiment of the present disclosure, the repairing unit 603 may perform repairing on the deformed region by filling the deformed region with background pixels. For example, the inpainting unit 603 may perform inpainting on the deformed region through an image inpainting algorithm or a background frame replacement method. Of course, the present disclosure is not limited to these repairing methods, and the repairing unit 603 may also use any possible repairing method to repair the deformed area. The following describes in detail how to perform inpainting on deformed regions through image inpainting algorithms or background frame replacement.

According to an exemplary embodiment of the present disclosure, the repairing unit 603 may fill the deformed region with background pixels using an image inpainting algorithm based on the deformed region and the first image or the first image. For example, image inpainting algorithms may include traditional inpainting algorithms (non-deep learning algorithms) and deep learning algorithms. Conventional patching algorithms may include patch-based methods and diffusion-based methods. When using the image block-based method, the repairing unit 603 may fill in the deformed area by searching for an image block similar to the deformed area on the first image before performing face-lifting. When the repairing unit 603 uses the diffusion-based method, the pixels at the edge of the deformed area may grow inward according to the properties of the corresponding area of the first image before performing face-lifting, and the entire deformed area may be filled by diffusion. The deep learning algorithm may include a convolutional neural network (CNN)-based method, a generative adversarial network (GAN)-based method, a recurrent neural network (RNN)-based method, and the like. The repairing unit 603 may generate a mask based on the deformed area, and perform image inpainting on the deformed area by using a deep learning algorithm based on the second image after performing face reduction and the generated mask. The specific method may be that the repairing unit 603 inputs the second image after face-lifting and the generated mask to the model based on the deep learning algorithm, and the model based on the deep learning algorithm outputs the third image after the deformed area is repaired.

According to an exemplary embodiment of the present disclosure, under the condition that a pure background image with the same scene as the first image can be obtained, the repairing unit 603 may use a simpler and faster background frame replacement method to repair the deformed region. For example, but not limited to, when the photographing device for photographing the first image is stationary and the position is fixed, the background image before adding the human face and the first image after adding the human face can be obtained respectively. For example, in a video scene, multiple frames of video images can be continuously captured, and the shooting device can first capture image frames with only background to obtain background frame images, and then allow the user to shoot in front of the shooting device to obtain video frame images (for example, the first image).

After acquiring the background image, the repairing unit 603 may fill the deformed area with background pixels based on the background image and the second image. For example, the repairing unit 603 may search for the same area as the deformed area in the background image, and replace the pixel value of the pixel in the deformed area with the pixel value of the pixel of the searched area in the background image.

FIG. 7 is a block diagram of an electronic device 700 according to an exemplary embodiment of the present disclosure.

Referring to FIG. 7 , the electronic device 700 includes at least one memory 701 and at least one processor 702. The at least one memory 701 stores a computer-executable instruction set. When the computer-executable instruction set is executed by the at least one processor 702, the execution An image processing method according to an exemplary embodiment of the present disclosure.

As an example, the electronic device 700 may be a PC computer, a tablet device, a personal digital assistant, a smart phone, or other device capable of executing the above set of instructions. Here, the electronic device 700 is not necessarily a single electronic device, but can also be a collection of any device or circuit capable of individually or jointly executing the above-mentioned instructions (or instruction sets). Electronic device 700 may also be part of an integrated control system or system manager, or may be configured as a portable electronic device that interfaces locally or remotely (eg, via wireless transmission).

In electronic device 700, processor 702 may include a central processing unit (CPU), graphics processing unit (GPU), programmable logic device, special purpose processor system, microcontroller, or microprocessor. By way of example and not limitation, processors may also include analog processors, digital processors, microprocessors, multi-core processors, processor arrays, network processors, and the like.

Processor 702 may execute instructions or code stored in memory 701, which may also store data. Instructions and data may also be sent and received over a network via a network interface device, which may employ any known transport protocol.

The memory 701 may be integrated with the processor 702, eg, RAM or flash memory arranged within an integrated circuit microprocessor or the like. Furthermore, memory 701 may comprise a separate device, such as an external disk drive, a storage array, or any other storage device that may be used by a database system. The memory 701 and the processor 702 may be operatively coupled, or may communicate with each other, eg, through I/O ports, network connections, etc., to enable the processor 702 to read files stored in the memory.

Additionally, the electronic device 700 may also include a video display (such as a liquid crystal display) and a user interaction interface (such as a keyboard, mouse, touch input device, etc.). All components of electronic device 700 may be connected to each other via a bus and/or network.

According to exemplary embodiments of the present disclosure, there may also be provided a computer-readable storage medium storing instructions, wherein the instructions, when executed by at least one processor, cause the at least one processor to perform a video demarking method according to the present disclosure. Examples of the computer-readable storage medium herein include: Read Only Memory (ROM), Random Access Programmable Read Only Memory (PROM), Electrically Erasable Programmable Read Only Memory (EEPROM), Random Access Memory (RAM) , dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, non-volatile memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM , DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, Blu-ray or Optical Disc Storage, Hard Disk Drive (HDD), Solid State Hard disk (SSD), card memory (such as a multimedia card, Secure Digital (SD) card, or Extreme Digital (XD) card), magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid state disk, and any other apparatus configured to store and provide the computer program and any associated data, data files and data structures in a non-transitory manner with the computer program and any associated data, data files and data structures The computer program is given to a processor or computer so that the processor or computer can execute the computer program. The computer program in the above-mentioned computer readable storage medium can be executed in an environment deployed in a computer device such as a client, a host, a proxy device, a server, etc. Furthermore, in one example, the computer program and any associated data, data files and data structures are distributed over networked computer systems so that the computer programs and any associated data, data files and data structures are stored, accessed and executed in a distributed fashion by one or more processors or computers.

According to an exemplary embodiment of the present disclosure, there can also be provided a computer program product, wherein instructions in the computer program product can be executed by a processor of a computer device to complete the image processing method according to the exemplary embodiment of the present disclosure.

Other embodiments of the present disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or techniques in the technical field not disclosed by the present disclosure . The specification and examples are to be regarded as exemplary only, with the true scope and spirit of the disclosure being indicated by the following claims.

It is to be understood that the present disclosure is not limited to the precise structures described above and illustrated in the accompanying drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

An image processing method, comprising:

identifying the face region in the first image;

Perform face-lifting on the face region in the first image based on a face-lifting algorithm to obtain a face-lifted second image;

Inpainting is performed on the deformed region in the second image resulting from performing the face-lifting to obtain the inpainted third image.
The image processing method according to claim 1, wherein the deformed area is a predetermined area including the face area involved in performing face reduction in the first image except the face area after face reduction Area.
The image processing method according to claim 1, wherein the performing restoration on the deformed region in the second image caused by performing face-lifting comprises:

Inpainting is performed by filling the deformed area with background pixels.
The image processing method according to claim 3, wherein the performing restoration by filling the deformed area with background pixels, comprising:

Based on the deformed region and the first image or first image, the deformed region is filled with background pixels using an image inpainting algorithm.
The image processing method according to claim 3, wherein the performing restoration by filling the deformed area with background pixels, comprising:

filling the deformed region with background pixels based on the background image and the second image,

Wherein, the background image is a pure background image with the same scene as the first image.
The image processing method according to claim 5, wherein, based on the background image and the second image, filling the deformed area with background pixels, comprising:

searching the background image for the same area as the deformed area;

The pixel values of the pixels in the deformed area are replaced with the pixel values of the pixels of the searched area in the background image.
An image processing device, comprising:

an identification unit configured to identify the face region in the first image;

A face-lifting unit, configured to perform face-lifting on the face region in the first image based on a face-lifting algorithm to obtain a face-lifted second image;

The inpainting unit is configured to perform inpainting on the deformed region in the second image resulting from performing the face reduction, so as to generate a repaired third image.
8. The image processing apparatus according to claim 7, wherein the deformed area is a predetermined area including the face area involved in performing face reduction in the first image except the face area after face reduction Area.
The image processing apparatus according to claim 7, wherein the repairing unit is configured to:

Inpainting is performed by filling the deformed area with background pixels.
The image processing apparatus of claim 9, wherein the repair unit is configured to:

Based on the deformed region and the first image or first image, the deformed region is filled with background pixels using an image inpainting algorithm.
The image processing apparatus of claim 9, wherein the repair unit is configured to:

filling the deformed region with background pixels based on the background image and the second image,

Wherein, the background image is a pure background image with the same scene as the first image.
The image processing apparatus of claim 11, wherein the repair unit is configured to:

searching the background image for the same area as the deformed area;

The pixel values of the pixels in the deformed area are replaced with the pixel values of the pixels of the searched area in the background image.
An electronic device, comprising:

at least one processor;

at least one memory storing computer-executable instructions,

Wherein, the computer-executable instructions, when executed by the at least one processor, cause the at least one processor to perform the image processing method of any one of claims 1-6.
A computer-readable storage medium storing instructions, wherein the instructions, when executed by at least one processor, cause the at least one processor to perform the method of any one of claims 1 to 6 image processing method.