WO2022193573A1 - Facial fusion method and apparatus - Google Patents

Abstract

A facial fusion method and apparatus, and an electronic device, a storage medium and a program product, which relate to the field of image processing. The facial fusion method comprises: respectively blurring an original facial image and a target facial image to obtain a first original facial blurred image and a first target facial blurred image; performing blurring processing on facial pure-white images respectively generated according to the original facial image and the target facial image, so as to obtain respective pure-white blurred images of the original facial image and the target facial image; performing image decomposition on the first original facial blurred image on the basis of the pure-white blurred image of the original facial image, so as to obtain a second original facial blurred image, and performing image decomposition on the first target facial blurred image on the basis of the pure-white blurred image of the target facial image, so as to obtain a second target facial blurred image; performing image decomposition on the original facial image on the basis of the second original facial blurred image, so as to obtain an original facial detail image; and fusing the original facial detail image with the second target facial blurred image, so as to obtain a facial fused image.

Description

Face fusion method and device

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is based on the Chinese patent application with the application number of 202110290169.5 and the filing date of March 18, 2021, and claims the priority of the Chinese patent application. The entire content of the Chinese patent application is incorporated herein by reference.

technical field

The present disclosure relates to the field of image processing, and in particular, to a face fusion method, apparatus, electronic device, computer-readable storage medium, and a computer program product.

Background technique

With the popularity of video applications such as short videos and live broadcasts, people have a need for various special effects editing for faces in videos. Among them, the face-changing special effect is a hot spot in various special effects editing. Face-changing special effects refer to replacing the face in one image with the identity features of another face, but keeping other features in the image, such as the background, characters' clothing, etc. unchanged. Therefore, the face-changing special effect is essentially an image. Synthetic processing.

The current face-changing special effects mainly detect the key points of the face contour, then determine the face contour according to the key points, and then replace the face with another face according to the face contour. Since there may be differences in human skin color, only part of the face is replaced, and edge color distortion will appear at the junction of the fused face and the unchanged part. For example, after replacing a face with a lighter complexion with a face with a darker complexion, there will be significant color differences between the face and the forehead and neck.

Color transfer technology appends the overall color of an image to another image, resulting in the image having the shape of the original image but with the color of the other image added.

SUMMARY OF THE INVENTION

The present disclosure provides a face fusion method, apparatus, electronic device, computer-readable storage medium, and a computer program product.

According to a first aspect of the embodiments of the present disclosure, there is provided a face fusion method, including:

Obtain the original face map and the target face map;

The original face image and the target face image are respectively subjected to a blurring process to obtain a first original face blurred image and a first target face blurred image;

Performing a blurring process on the pure white face image generated according to the original face image, to obtain a pure white blurred image of the original face image, and performing a blurring process on the pure white face image generated according to the target face image The image is blurred to obtain a pure white blurred image of the target face image;

Based on the pure white blurred image of the original face image, image decomposition is performed on the first original face blurred image to obtain a second original face blurred image, and a pure white blurred image based on the target face image , performing image decomposition on the first target face blurred image to obtain a second target face blurred image;

Based on the second original face blurred image, image decomposition is performed on the original face image to obtain an original face detail image; and

The original face detail image is fused to the second target face blur image to obtain a face fusion image of the target face image.

In an exemplary embodiment, performing image decomposition on the first original blurred face image based on the pure white blurred image of the original face image to obtain a second original blurred face image, including:

Normalizing the pixel value of each pixel in the pure white blurred image of the original face image to obtain a first normalized blurred image;

Divide the pixel value of each pixel in the first original face blur map by the pixel value of each pixel in the first normalized blur map to obtain the second original face blur map.

In an exemplary embodiment, performing normalization processing on the pixel values of each pixel in the pure white blurred image of the original face image to obtain a first normalized blurred image, including:

Divide the pixel value of each pixel in the pure white fuzzy image of the original face image by the pure white pixel value to obtain the first normalized fuzzy image; the pure white pixel value is the pixel value of the pure white pixel .

In an exemplary embodiment, based on the pure white blurred image of the target face image, image decomposition is performed on the first target face blurred image to obtain a second target face blurred image, including:

Normalizing the pixel value of each pixel in the pure white blur image of the target face image to obtain a second normalized blur image;

Divide the pixel value of each pixel point in the first target face blur map by the pixel value of each pixel point in the second normalized blur map to obtain the second target face blur map.

In an exemplary embodiment, the pixel values of each pixel in the pure white blur image of the target face image are normalized to obtain a second normalized blur image, including:

The second normalized blur image is obtained by dividing the pixel value of each pixel in the pure white blur image of the target face image by the pure white pixel value.

In an exemplary embodiment, performing image decomposition on the original face image based on the second original face blur image to obtain an original face detail image, including:

dividing the pixel value of each pixel in the original face image by the pixel value of each pixel in the second original face blur image to obtain the original face detail image;

The fusion of the original face detail image to the second target face blur image to obtain a face fusion image of the target face image includes:

Multiply the pixel value of each pixel in the original face detail map by the pixel value of each pixel in the second target face blur map to obtain a face fusion map of the target face map.

In an exemplary embodiment, performing image decomposition on the original face image based on the second original face blur image to obtain an original face detail image, including:

Subtracting the pixel value of each pixel in the original face image from the pixel value of each pixel in the second original face blur image to obtain the original face detail image;

The fusion of the original face detail image to the second target face blur image to obtain a face fusion image of the target face image includes:

The pixel value of each pixel point in the original face detail map is added to the pixel value of each pixel point in the second target face blur map to obtain a face fusion map of the target face map.

According to a second aspect of the embodiments of the present disclosure, there is provided a face fusion device, including:

an acquisition unit, configured to acquire the original face map and the target face map;

a face blurring unit, configured to blur the original face image and the target face image respectively to obtain a first original face blurred image and a first target face blurred image;

The pure white blurring unit is configured to perform a blurring process on the pure white image of the human face generated according to the original human face image to obtain a pure white blurred image of the original human face image, and, The pure white image of the face generated by the image is subjected to fuzzy processing to obtain the pure white blurred image of the target face image;

a blurring processing unit, configured to perform image decomposition on the first original face blurred image based on the pure white blurred image of the original face image to obtain a second original face blurred image, and, based on the target person The pure white blurred image of the face image, the first target face blurred image is decomposed to obtain the second target face blurred image;

a detail decomposition unit, configured to perform image decomposition on the original face map based on the second original face blur map to obtain an original face detail map; and

The face fusion unit is configured to fuse the original face detail image to the second target face fuzzy image to obtain a face fusion image of the target face image.

In an exemplary embodiment, the obfuscation unit is configured to:

Normalizing the pixel value of each pixel in the pure white blurred image of the original face image to obtain a first normalized blurred image;

Divide the pixel value of each pixel in the first original face blur map by the pixel value of each pixel in the first normalized blur map to obtain the second original face blur map.

In an exemplary embodiment, the obfuscation unit is further configured to:

Divide the pixel value of each pixel in the pure white fuzzy image of the original face image by the pure white pixel value to obtain the first normalized fuzzy image; the pure white pixel value is the pixel value of the pure white pixel .

In an exemplary embodiment, the obfuscation unit is configured to:

Normalizing the pixel value of each pixel in the pure white blur image of the target face image to obtain a second normalized blur image;

Divide the pixel value of each pixel point in the first target face blur map by the pixel value of each pixel point in the second normalized blur map to obtain the second target face blur map.

In an exemplary embodiment, the obfuscation unit is further configured to:

The second normalized blur image is obtained by dividing the pixel value of each pixel in the pure white blur image of the target face image by the pure white pixel value.

In an exemplary embodiment, the detail decomposition unit is configured to:

Divide the pixel value of each pixel in the original face image by the pixel value of each pixel in the second original face blur to obtain the original face detail image;

The face fusion unit is configured as:

Multiply the pixel value of each pixel in the original face detail map by the pixel value of each pixel in the second target face blur map to obtain a face fusion map of the target face map.

In an exemplary embodiment, the detail decomposition unit is configured to:

Subtracting the pixel value of each pixel in the original face image from the pixel value of each pixel in the second original face blur image to obtain the original face detail image;

The face fusion unit is configured as:

The pixel value of each pixel point in the original face detail map is added to the pixel value of each pixel point in the second target face blur map to obtain a face fusion map of the target face map.

According to a third aspect of the embodiments of the present disclosure, an electronic device is provided, including a memory and a processor, the memory stores a computer program, and the processor implements the first aspect or the first aspect when executing the computer program Any possible implementation of the described face fusion method.

According to a fourth aspect of the embodiments of the present disclosure, a computer-readable storage medium is provided, when the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device can execute the first aspect Or the face fusion method of any one of the first aspect.

According to a fifth aspect of the embodiments of the present disclosure, a computer program product is provided, the computer program product includes a computer program, and when the computer program is executed by a processor, the face fusion according to any one of the first aspects is implemented method.

By blurring the original face image and the target face image respectively, the first original face blur image and the first target face blur image which are composed of low-frequency signals and retain the original skin color of the face are obtained. The pure white face image generated by the original face image and the target face image is blurred to obtain the respective pure white blurred image, and the first original face blurred image and the first target face blurred image are processed by using the pure white blurred image. The image is decomposed to eliminate the black edges of the face edges in the first original face blurred image and the first target face blurred image due to blurring processing, and obtain the second original face blurred image and the second target face blurred image , and then decompose the original face image through the second original face blur image to obtain the original face detail image, and finally fuse the original face detail image into the second target face blur image to obtain the target face image. Face fusion map. Therefore, the fusion of the facial features in the face image and the original skin color of another face image is achieved in a simple manner, without the need to perform a large number of color migration operations, and the processing efficiency is high. Therefore, color distortion due to blurring processing is avoided, and the speed of face fusion is improved at the same time.

It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure.

Description of drawings

The accompanying drawings, which are incorporated into and constitute a part of this specification, illustrate embodiments consistent with the present disclosure, and together with the description, serve to explain the principles of the present disclosure and do not unduly limit the present disclosure.

FIG. 1A shows a schematic diagram of an initial human face image that has not been processed by the special effect of face-changing.

FIG. 1B shows a schematic diagram of a face-changing effect.

Fig. 2 is a flow chart of a method for face fusion according to an exemplary embodiment.

FIG. 3A shows a schematic diagram of a face image to be subjected to face swapping.

FIG. 3B shows a schematic diagram of a face blur map.

FIG. 3C shows a schematic diagram of black borders appearing at the edge of a human face in a blurred image of a human face.

FIG. 3D shows a schematic diagram of a pure white image of a human face.

Figure 3E shows a schematic diagram of a pure white blur map.

FIG. 3F shows a schematic diagram of a face detail map.

FIG. 3G shows a schematic diagram of a face fusion map for performing face fusion based on high and low frequency signals of an image.

Fig. 4 is a flowchart showing a step of processing a face blurred image by using a pure white blurred image according to an exemplary embodiment.

Fig. 5 is a block diagram of a face fusion apparatus according to an exemplary embodiment.

Fig. 6 is an internal structure diagram of an electronic device according to an exemplary embodiment.

Detailed ways

In order to make those skilled in the art better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings.

It should be noted that the terms "first", "second" and the like in the description and claims of the present disclosure and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific sequence or sequence. It is to be understood that the data so used may be interchanged under appropriate circumstances such that the embodiments of the disclosure described herein can be practiced in sequences other than those illustrated or described herein. The implementations described in the illustrative examples below are not intended to represent all implementations consistent with this disclosure.

The face fusion method provided by the present disclosure can be applied to the application environment of editing face-changing special effects through a terminal. Wherein, the terminal may be, but not limited to, various personal computers, notebook computers, smart phones, tablet computers and portable wearable devices. The face fusion method provided by the present disclosure can be applied to various application scenarios that require editing of special effects for changing faces. In one of the scenarios, the user can edit the face-changing special effects for more than two faces in the video while shooting the video, obtain the face-changing video and upload it to the video sharing platform for other users to watch. In another scenario, the user can edit the face-changing special effects in real time under the condition of live video. The user can also photograph a face through the terminal, and edit the face-changing special effect between the photographed face and the face of another image. The user can also shoot two faces through the terminal, and edit the face-changing special effects on the two faces obtained by shooting. Those skilled in the art can apply the face fusion method provided by the present disclosure to various application scenarios for editing face-changing special effects according to the actual situation.

FIG. 1A shows a schematic diagram of an initial human face image that has not been processed by the special effect of face-changing. As can be seen from the figure, the initial face image contains the original face on the left and the target face on the right. The face-changing effect is performed on the original face and the target face, and the face-changing effect shown in Fig. 1B is obtained. However, the skin color of the left face is different from that of the right face. After covering the left face on the right face, at the junction between the new face and the uncovered part of the original face On the upper and lower sides, there is a color difference as shown in the area 101 marked in FIG. 1B , that is, there is edge color distortion. Improving the edge through color migration technology will seriously affect the face-changing efficiency.

Fig. 2 is a flow chart of a face fusion method according to an exemplary embodiment. As shown in Fig. 2 , the face fusion method of the present disclosure includes the following steps.

In step S210, the original face map and the target face map are obtained.

The face map may be an image containing a human face. The original face and the target face can be two face swap objects for face swap effect editing.

In some embodiments, in different application scenarios, the terminal may obtain the above-mentioned original face map and target face map in corresponding different ways. For example, in a scenario where a user edits a face-changing special effect on a face in a video, the user can select a video on the terminal, select two faces in the video that need face-changing, and submit a face-changing request. The terminal extracts images containing the two faces in the video frame respectively according to the face-changing request, as the above-mentioned original face image and target face image respectively. Those skilled in the art can determine the means for obtaining the original face image and the target face image according to the actual application scenario.

FIG. 3A shows a schematic diagram of a face image to be subjected to face swapping. As can be seen from the figure, two face images containing the left and right sides to be swapped can be extracted from an image. For the convenience of explanation, the face image on the left is named as the original face image, and correspondingly, the face image on the right is named as the target face image.

In step S220, blurring is performed on the original face image and the target face image respectively, to obtain a first original face blurred image and a first target face blurred image.

Among them, blurring processing is an image processing method used to obtain low-frequency signals in an image and make the image blurred. Common blur processing mainly includes Gaussian blur, a data smoothing technique based on Gaussian distribution. The principle is to take the average value of the surrounding pixels for each pixel in the image, so that the image loses details.

In practical applications, the above-mentioned Gaussian blurring can be achieved by the following two-dimensional Gaussian function:

Among them, (x, y) is the coordinate of a certain pixel, G is the blurred pixel value of the pixel (x, y), and σ represents the degree of smoothness.

The face blur map may be an image composed of low-frequency signals and used to express the basic color of the face. It should be noted that, from the perspective of the degree of change in image brightness or grayscale, the image contains high-frequency signals and low-frequency signals. The low-frequency signal of the image represents the area where the brightness or grayscale changes slowly in the image, that is, the area where the color changes less and is relatively flat in the image, and the low-frequency signal usually describes the main content of the image. The high-frequency signal represents the area where the brightness or grayscale changes sharply in the image, that is, the area in the image where the color changes greatly, and the edge contour and detail features are displayed.

In some embodiments, the terminal may perform Gaussian blurring on the original face image and the target face image respectively. After Gaussian blurring, since the color dithering frequency of each color channel in the image is low, the blurred image is composed of low-frequency signals. The low-frequency signal can reflect the basic skin color of the human face. The face image after Gaussian blurring eliminates the high-frequency signals representing the facial features of the face, but retains the image representing the color of the face and the low-frequency signal with obvious bright contrast. This image is the above-mentioned blurred face. picture.

FIG. 3B shows a schematic diagram of a face blur map. As can be seen from the figure, after Gaussian blurring is performed on the original face image and the target face image in Fig. 3A respectively, two blurred faces on the left and right in Fig. 3B are obtained, namely the first original face blurred image and the first target. Blurred human face. The color and brightness of each pixel in the blurred image have a low frequency of change and are all low-frequency signals. By blurring the face image, the high-frequency signals representing the detailed features of the facial features of the face are removed, and an image composed of low-frequency signals, representing the colors in the face and with obvious bright contrast, is obtained as a face blur map.

In step S230, blurring the pure white face image generated according to the original face image to obtain a pure white blurred image of the original face image, The pure white image of the target face is blurred to obtain the pure white blurred image of the target face image.

It should be noted that the original face image is directly divided by the first original face blur image to obtain the original face detail image, and then the original face detail image and the first target face blur image are multiplied to realize the human face. Face fusion may cause color distortion at the edges of the fused face. After in-depth research by the applicant, it is found that although the Gaussian blur method can save the calculation amount of color migration, however, since the Gaussian blur has a certain blur radius, in the case of blur, the generated pixel values outside the face range tend to be 0 pixels, thus forming a circle of black borders at the edge of the face, as shown in Figure 3C, a schematic diagram of the presence of black borders on the edge of the face in the blurred face image, the edge of the face in the first original blurred face image There is a black border around the face. Therefore, the fusion based on the blurred face image with black borders will cause the edge color distortion of the final fused face. Therefore, the edge color distortion of the fusion face can be avoided by introducing a pure white image method.

In some embodiments, the terminal may first generate a corresponding pure-white face image according to the original face image, and generate a corresponding pure-white face image according to the target face image.

FIG. 3D shows a schematic diagram of a pure white image of a human face. As can be seen from the figure, the image on the right is a pure white face image generated from the original face image on the left. The pure white face image has a solid color area filled with pure white with the same grayscale. The edge of the region matches the face in the original face image on the left in terms of shape, size and other features.

Then, the pure white face images of the original face image and the target face image are respectively blurred to obtain the pure white blurred image of the original face image and the pure white blurred image of the target face image.

Figure 3E shows a schematic diagram of a pure white blurred image. It can be seen from the figure that Gaussian blurring is performed on the pure white image of the face on the left, and the pure white blurred image on the right is obtained, and the edges in the pure white blurred image are blurred.

In step S240, based on the pure white blurred image of the original face image, image decomposition is performed on the first original face blurred image to obtain a second original face blurred image, and, based on the target face image The pure white blurred image of the first target face is decomposed to obtain the second target face blurred image.

In some embodiments, the terminal may first perform normalization processing on the pixel values of each pixel in the pure white blurred images of the original face image and the target face image, respectively, to obtain the original face image and the target face image respectively. Then, divide the pixel value of each pixel in the original face image and the target face image by the pixel value of each pixel in the corresponding normalized blur image to obtain The above-mentioned second original face blurred image and second target face blurred image.

By using the pure white blur image to decompose the first face blur image to obtain the second face blur image, the pixel values of the pixels outside the face edge (black border) in the original face blur image can be changed, Change its pixel value to a pixel value close to the pixel value of the pixel point on the edge of the face. As a result, the black edge of the face edge in the face blurred image disappears, and after the subsequent face fusion processing, the edge color distortion of the fused face is avoided.

In step S250, based on the second original face blurred image, image decomposition is performed on the original face image to obtain an original face detail image.

The face detail map may be an image composed of high-frequency signals and used to express the detailed features of the facial features of the face.

In some embodiments, the terminal may perform image decomposition on the original face image by using the second original face blurred image, so as to obtain an image composed of high-frequency signals and used to express the detailed features of the facial features of the human face. Detail of the face.

It should be noted that usually an image can be composed of a detail layer and a blur layer, and the detail layer can be obtained by decomposing the blur layer on the image. There can be various implementations of image decomposition, for example, the blur layer can be removed from the original image to obtain the detail layer based on multiplication decomposition or additive decomposition. In some embodiments, image decomposition is performed by means of additive decomposition, the grayscale of each pixel in the face image can be extracted to form a grayscale matrix, and the grayscale of each pixel in the face blur layer can be extracted to form another grayscale By subtracting the two grayscale matrices in alignment, the resulting grayscale matrix can constitute the face detail layer.

FIG. 3F shows a schematic diagram of a face detail map. It can be seen from the figure that the detailed face image decomposed by removing the blurred image of the face retains the texture and contour details of the facial features such as eyebrows, eyes, nose, mouth, etc., but the original facial features of the face have been removed. complexion.

In step S260, the original face detail image is fused to the second target face fuzzy image to obtain a face fusion image of the target face image.

The face fusion image may be an image obtained by merging the facial features of one face with the facial features of another face except the facial features.

In some embodiments, the terminal may fuse the original face detail image and the second target face blur image into the above-mentioned face fusion image based on multiplicative fusion or additive fusion. For example, by performing image fusion by means of additive fusion, the grayscale matrix formed by the grayscale of each pixel in the original face detail image can be combined with the grayscale formed by each pixel in the second target face blurred image. The grayscale matrix is added, and the obtained grayscale matrix can constitute the above-mentioned face fusion map.

FIG. 3G shows a schematic diagram of a face fusion map for performing face fusion based on high and low frequency signals of an image. As can be seen from the figure, by fusing the original face detail image with the second target face blur image, the obtained face fusion image, the face on the right retains the basic skin color of the original target face, but The facial features of the original face image on the left are integrated. Even if there is a difference in skin color, the facial features of another face are fused on the basis of the original skin color of the face during the fusion process. Therefore, after the left face is fused to the right face, the area 301 in the figure does not appear. Color differences appear, no edge color distortion. Moreover, by using the pure white blurred image to eliminate the black edges generated by the blurring of the edge of the face, there is no color distortion on the left and right faces in the face fusion image.

It should be noted that, in the case of changing faces, it is also necessary to fuse the facial features of the target face image into the original face map through the above-mentioned face fusion method to obtain a face fusion map of another face. That is, a complete face-changing special effect processing requires at least two face fusions through the above-mentioned face fusion method. Since the fusion process of another face is similar to the above-mentioned embodiment, only the fusion objects are different, those skilled in the art can clearly know the complete special effect processing method of face-changing according to the above-mentioned face fusion method, which is not repeated here. Repeat.

In the above-mentioned face fusion method, by blurring the original face image and the target face image respectively, the first original face blur image and the first target which are composed of low-frequency signals and retain the original skin color of the face are obtained. The face blurred image, and then the pure white face image generated according to the original face image and the target face image is blurred to obtain the respective pure white blurred image, and the first original face blurred image and The first target face blurred image is decomposed to eliminate the black edges of the face edges in the first original face blurred image and the first target face blurred image due to blurring processing, and the second original face blurred image is obtained. and the second target face blur map, and then decompose the original face map through the second original face blur map to obtain the original face detail map, and finally fuse the original face detail map into the second target face blur map, The face fusion map of the target face map is obtained. Therefore, the fusion of the facial features in the face image and the original skin color of another face image is achieved in a simple manner, without the need to perform a large number of color migration operations, and the processing efficiency is high. Therefore, color distortion due to blurring processing is avoided, and the speed of face fusion is improved at the same time.

In an exemplary embodiment, as shown in FIG. 4 , in step S240, the following steps may be used:

In step S441, normalize the pixel values of each pixel in the pure white blurred image of the original face image to obtain a first normalized blurred image; The pixel value of each pixel is divided by the pixel value of each pixel in the first normalized blur map to obtain the second original face blur map.

In some embodiments, after the pure white blur image of the original face image is obtained, the pixel values of each pixel in the pure white blur image are normalized to obtain the first normalized blur image. For example, divide the pixel value of each pixel in the pure white blur image by the pixel value of the pure white pixel by 255 (the pixel value of the pure white pixel is the theoretical highest value), so as to divide the pixel value of each pixel in the pure white blur image The value is normalized, thereby obtaining the first normalized blur image generated for the pure white blur image of the original face image.

Then, the pixel value of each pixel in the first original face blur map is divided by the pixel value of each pixel in the first normalized blur map to obtain the above-mentioned second original face blur map.

In step S442, normalize the pixel values of each pixel in the pure white fuzzy map of the target face map to obtain a second normalized fuzzy map; The pixel value of each pixel is divided by the pixel value of each pixel in the second normalized blur map to obtain the second target face blur map.

In some embodiments, after the pure white blurred image of the original face image is obtained, the same steps as the pure white blurred image of the original face image are used to obtain the second target face blurred image, which will not be repeated here. .

In the above face fusion method, a normalized blur image is obtained based on normalizing each pixel point in the pure white blur image, and then the pixel values of the pixels in the face blur image are divided by the normalized blur image. The pixel values of each pixel point are used to obtain the second original face blur map and the second target face blur map, and subsequent large-scale numerical operations based on the normalized values can effectively reduce the amount of calculation. The processing can eliminate the black edges in the blurred face image to avoid the edge color distortion of the fusion face, which improves the speed of face fusion while ensuring the fusion quality of the face fusion image.

In an exemplary embodiment, in step S441, the pixel value of each pixel in the pure white blur image of the original face image is normalized to obtain a first normalized blur image, which can be obtained by the following method: Steps to achieve:

Divide the pixel value of each pixel in the pure white fuzzy image of the original face image by the pure white pixel value to obtain the first normalized fuzzy image; the pure white pixel value is the pixel value of the pure white pixel ;

In step S442, the pixel value of each pixel in the pure white blurred image of the target face image is normalized to obtain a second normalized blurred image, which can be achieved through the following steps:

The second normalized blur image is obtained by dividing the pixel value of each pixel point in the pure white blur image of the target face image by the pure white pixel value.

Table 1 shows the pixel points A and B on the original face map, the pixel points C at the edge of the face, and the pixels D and E outside the edge of the face. RGB three channels are used to represent the pixel value of each pixel point.

	A	B	C	D	E
R	204	206	160	/	/
G	148	154	108	/	/
B	120	132	86	/	/

Table 1

After the original face image is blurred, the pixel point data of the first original face blurred image in Table 2 is obtained:

	A	B	C	D	E
R	102	103	80	64	46
G	74	74	54	43	31
B	60	60	43	34	25

Table 2

It can be seen from Table 2 that in the case of Gaussian blurring, the pixels D and E within the blur radius of the pixel C are given a certain pixel value (usually close to 0), the pixel D and E is the black edge at the edge of the face in the blurred face. This black border can be eliminated by introducing a pure white blur image. Table 3 shows the pixel point data of the pure white face image generated from the original face blurred image:

	A	B	C	D	E
R	255	255	255	/	/
G	255	255	255	/	/
B	255	255	255	/	/

table 3

Gaussian blur is performed on each pixel in the pure white face image, and the pixel data of the pure white blurred image shown in Table 4 is obtained:

	A	B	C	D	E
R	127	127	127	102	73
G	127	127	127	102	73
B	127	127	127	102	73

Table 4

Next, normalize the pixel value of each pixel in the pure white blur map shown in Table 4, that is, divide each pixel by the pixel value of the pure white pixel 255, and obtain the normalization in Table 5 below Pixel point data of the blurred image:

	A	B	C	D	E
R	0.498	0.498	0.498	0.4	0.286
G	0.498	0.498	0.498	0.4	0.286
B	0.498	0.498	0.498	0.4	0.286

table 5

Finally, divide the pixel value of the pixel point of the first original face blurred image shown in Table 2 by the pixel value of the pixel point of the normalized blurred image shown in Table 5 to obtain the second image shown in Table 6 below. Pixel data of the original face blur map:

	A	B	C	D	E
R	204	206	160	160	161
G	148	154	108	107	108
B	120	132	86	85	87

Table 6

It can be seen from the pixel data of the second original face blurred image in Table 6 that the pixel values of pixels D and E of the second original face blurred image are close to the pixel value of pixel C, and the black borders are transformed into Pixels with similar pixel values at the edge of the face, thus eliminating the black edges caused by Gaussian blur by introducing pure white blur.

In the above-mentioned face fusion method, the pixel value of the pure white pixel is used to normalize the pixel value of each pixel in the pure white blur image to obtain the first normalized blur image and the second normalized blur image. , the normalization can be completed without complex numerical transformation processing, which improves the speed of face fusion.

In an exemplary embodiment, in step S250, the following steps may be used:

dividing the pixel value of each pixel in the original face image by the pixel value of each pixel in the second original face blur image to obtain the original face detail image;

In step S260, it can be achieved by the following steps:

Multiply the pixel value of each pixel in the original face detail map by the pixel value of each pixel in the second target face blur map to obtain a face fusion map of the target face map.

In some embodiments, when the image is decomposed by means of multiplicative decomposition, the pixel value of each pixel in the original face image (for example, the pixel value of three RGB channels) can be extracted first, and the pixel value of each pixel in the image can be extracted according to the The coordinate position of the original face image is established, and the pixel value matrix of the original face image is established. Then, the pixel value of each pixel point in the second original face blur map can be extracted, and the pixel value matrix of the second original face blur map is established according to the coordinate position of each pixel point in the image. Then, the two pixel value matrices are divided, that is, each pixel value is divided according to the coordinate position of each pixel in the image, and the original face details reflecting the facial features of the original face map are obtained. picture.

In the case of fusing images by means of multiplicative decomposition, multiply the pixel value matrix of the original face detail map with the pixel value matrix of the second target face blur map, and then generate the pixel value matrix obtained by the multiplication. The above face fusion map.

In practical applications, the above face fusion method based on multiplicative decomposition can be implemented by the following algorithms:

dst=(dst _ori /dst _low )*source _low ;

dst _low = dst _blur /dst White _blur ;

source _low = source _blur /sourceWhite _blur ;

Among them, dst represents the final output face fusion image; dst _ori represents the original face image; dst _blur represents the first original face blur image; dstWhite _blur represents the pure white image obtained by blurring the original face image. Pure white blur image; dst _low represents the second original face blur image of the original face image; source _blur represents the first target face blur image; sourceWhite _blur represents the pure white image of the target face image obtained after blurring The pure white blur map of ; source _low represents the second target face blur map.

In the above-mentioned face fusion method, face fusion is performed by means of multiplicative decomposition. Compared with the method of additive decomposition, the original face color and facial features can be preserved as much as possible, and the quality of face fusion is better.

In an exemplary embodiment, in step S250, the following steps may be used:

Subtracting the pixel value of each pixel in the original face image from the pixel value of each pixel in the second original face blur image to obtain the original face detail image;

In step S260, it can be achieved by the following steps:

The pixel value of each pixel point in the original face detail map is added to the pixel value of each pixel point in the second target face blur map to obtain a face fusion map of the target face map.

In some embodiments, when the image is decomposed by means of additive decomposition, the pixel value of each pixel in the original face image (for example, the pixel value of three RGB channels) can be extracted first, and the pixel value of each pixel in the image can be extracted according to the The coordinate position of the original face image is established, and the pixel value matrix of the original face image is established. Then, the pixel value of each pixel in the second original face blur image can be extracted, and a pixel value matrix of the second original face blur image is established according to the coordinate position of each pixel in the image. Then, the two pixel value matrices are subtracted, that is, each pixel value is subtracted according to the coordinate position of each pixel in the image, and the original face detail map that reflects the facial features of the original face map is obtained. .

In the case of using additive decomposition to fuse images, the pixel value matrix of the original face detail map is added to the pixel value matrix of the second target face blur map, and the resulting pixel value matrix can be generated according to the addition. The above face fusion map.

In practical applications, the above-mentioned face fusion method based on additive decomposition can be implemented by the following algorithms:

dst=(dst _ori -dst _blur )+source _blur ;

dst _low = dst _blur /dst White _blur ;

source _low = source _blur /sourceWhite _blur ;

In the above-mentioned face fusion method, the face fusion is carried out by means of additive decomposition. Compared with the method of multiplicative decomposition, the required amount of computation is less, and the face fusion can be completed faster, and the efficiency of face fusion can be improved. higher.

It should be understood that although the steps in the flowcharts of FIGS. 2-7 are shown in sequence according to the arrows, these steps are not necessarily executed in the sequence shown by the arrows. Unless explicitly stated herein, the execution of these steps is not strictly limited to the order, and these steps may be performed in other orders. Moreover, at least a part of the steps in FIGS. 2-7 may include multiple steps or multiple stages. These steps or stages are not necessarily executed and completed at the same time, but may be executed at different times. The execution of these steps or stages The order is also not necessarily sequential, but may be performed alternately or alternately with other steps or at least a portion of the steps or phases within the other steps.

Fig. 5 is a block diagram of a face fusion apparatus according to an exemplary embodiment. 5 , the apparatus includes an acquisition unit 502 , a face blurring unit 504 , a pure white blurring unit 506 , a blurring processing unit 508 , a detail decomposition unit 510 and a face fusion unit 512 .

an obtaining unit 502, configured to obtain the original face map and the target face map;

The face blurring unit 504 is configured to blur the original face image and the target face image respectively to obtain the first original face blurred image and the first target face blurred image;

The pure white blurring unit 506 is configured to perform a blurring process on the pure white face image generated according to the original face image to obtain a pure white blurred image of the original face image, and The pure white image of the face generated by the face image is subjected to fuzzy processing to obtain the pure white blurred image of the target face image;

The blurring processing unit 508 is configured to perform image decomposition on the first original face blurred image based on the pure white blurred image of the original face image to obtain a second original face blurred image, and, based on the target The pure white blurred image of the face image, the first target face blurred image is decomposed to obtain the second target face blurred image;

The detail decomposition unit 510 is configured to perform image decomposition on the original face map based on the second original face blur map to obtain the original face detail map;

The face fusion unit 512 is configured to fuse the original face detail image to the second target face fuzzy image to obtain a face fusion image of the target face image.

In an exemplary embodiment, the blurring unit 508 is configured to:

Normalize the pixel value of each pixel in the pure white blurred image of the original face image to obtain a first normalized blurred image; The value is divided by the pixel value of each pixel in the first normalized blur map to obtain the second original face blur map; the pixel value of each pixel in the pure white blur map of the target face map is Perform normalization processing to obtain a second normalized fuzzy map; divide the pixel value of each pixel in the first target face fuzzy map by the pixel value of each pixel in the second normalized fuzzy map , to obtain the second target face blur map.

In an exemplary embodiment, the detail decomposition unit 510 is configured to:

dividing the pixel value of each pixel in the original face image by the pixel value of each pixel in the second original face blur image to obtain the original face detail image;

The face fusion unit 512 is configured as:

Multiply the pixel value of each pixel in the original face detail map by the pixel value of each pixel in the second target face blur map to obtain a face fusion map of the target face map.

In an exemplary embodiment, the detail decomposition unit 510 is configured to:

Subtracting the pixel value of each pixel in the original face image from the pixel value of each pixel in the second original face blur image to obtain the original face detail image;

The face fusion unit 512 is configured as:

The pixel value of each pixel point in the original face detail map is added to the pixel value of each pixel point in the second target face blur map to obtain a face fusion map of the target face map.

In an exemplary embodiment, the blurring unit 508 is configured to:

Divide the pixel value of each pixel in the pure white fuzzy image of the original face image by the pure white pixel value to obtain the first normalized fuzzy image; the pure white pixel value is the pixel value of the pure white pixel ; Divide the pixel value of each pixel in the pure white blurred image of the target face image by the pure white pixel value to obtain the second normalized blurred image.

Regarding the apparatus in the above-mentioned embodiments, the implementation manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be described in detail here.

FIG. 6 is a block diagram of an electronic device 600 for face fusion according to an exemplary embodiment. For example, electronic device 600 may be a mobile phone, computer, digital broadcast terminal, messaging device, game console, tablet device, medical device, fitness device, personal digital assistant, or the like.

6, electronic device 600 may include one or more of the following components: processing component 602, memory 604, power supply component 606, multimedia component 608, audio component 610, input/output (I/O) interface 612, sensor component 614 and communication component 616 .

The processing component 602 generally controls the overall operation of the electronic device 600, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 602 may include one or more processors 620 to execute instructions to perform all or some of the steps of the methods described above. Additionally, processing component 602 may include one or more modules that facilitate interaction between processing component 602 and other components. For example, processing component 602 may include a multimedia module to facilitate interaction between multimedia component 608 and processing component 602.

Memory 604 is configured to store various types of data to support operation at electronic device 600 . Examples of such data include instructions for any application or method operating on electronic device 600, contact data, phonebook data, messages, pictures, videos, and the like. Memory 604 may be implemented by any type of volatile or nonvolatile storage device or combination thereof, such as static random access memory (SRAM), electrically erasable programmable read only memory (EEPROM), erasable Programmable Read Only Memory (EPROM), Programmable Read Only Memory (PROM), Read Only Memory (ROM), Magnetic Memory, Flash Memory, Magnetic or Optical Disk.

Power supply assembly 606 provides power to various components of electronic device 600 . Power supply components 1006 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to electronic device 600.

Multimedia component 608 includes a screen that provides an output interface between the electronic device 600 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). In the case where the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touch, swipe, and gestures on the touch panel. The touch sensor may not only sense the boundaries of a touch or swipe action, but also detect the duration and pressure associated with the touch or swipe action. In some embodiments, the multimedia component 608 includes a front-facing camera and/or a rear-facing camera. When the electronic device 600 is in an operation mode, such as a shooting mode or a video mode, the front camera and/or the rear camera may receive external multimedia data. Each of the front and rear cameras can be a fixed optical lens system or have focal length and optical zoom capability.

Audio component 610 is configured to output and/or input audio signals. For example, audio component 610 includes a microphone (MIC) that is configured to receive external audio signals when electronic device 600 is in operating modes, such as calling mode, recording mode, and voice recognition mode. The received audio signal may be further stored in memory 604 or transmitted via communication component 616 . In some embodiments, audio component 610 also includes a speaker for outputting audio signals.

The I/O interface 612 provides an interface between the processing component 602 and a peripheral interface module, which may be a keyboard, a click wheel, a button, or the like. These buttons may include, but are not limited to: home button, volume buttons, start button, and lock button.

Sensor assembly 614 includes one or more sensors for providing status assessment of various aspects of electronic device 600 . For example, the sensor assembly 614 can detect the open/closed state of the electronic device 600, the relative positioning of the components, such as the display and the keypad of the electronic device 600, and the sensor assembly 614 can also detect the electronic device 600 or one of the electronic devices 600. Changes in the positions of components, presence or absence of user contact with the electronic device 600 , orientation or acceleration/deceleration of the electronic device 600 and changes in the temperature of the electronic device 600 . Sensor assembly 614 may include a proximity sensor configured to detect the presence of nearby objects in the absence of any physical contact. Sensor assembly 614 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 614 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

Communication component 616 is configured to facilitate wired or wireless communication between electronic device 600 and other devices. Electronic device 600 may access wireless networks based on communication standards, such as WiFi, carrier networks (eg, 2G, 3G, 4G, or 5G), or a combination thereof. In one exemplary embodiment, the communication component 616 receives broadcast signals or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 616 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

In an exemplary embodiment, electronic device 600 may be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable A programmed gate array (FPGA), controller, microcontroller, microprocessor or other electronic component implementation is used to perform the above method.

In an exemplary embodiment, there is also provided a non-transitory computer readable storage medium including instructions, such as memory 604 including instructions, executable by the processor 620 of the electronic device 600 to perform the above method. For example, the non-transitory computer-readable storage medium may be ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage device, and the like.

In an exemplary embodiment, a computer program product is also provided, including a computer program, wherein the computer program, when executed by a processor, completes the above-mentioned face fusion method.

All the embodiments of the present disclosure can be implemented independently or in combination with other embodiments, which are all regarded as the protection scope required by the present disclosure.

Claims (17)

1. A face fusion method, including:

   Obtain the original face map and the target face map;

   The original face image and the target face image are respectively subjected to a blurring process to obtain a first original face blurred image and a first target face blurred image;

   Performing a blurring process on the pure white face image generated according to the original face image, to obtain a pure white blurred image of the original face image, and performing a blurring process on the pure white face image generated according to the target face image The image is blurred to obtain a pure white blurred image of the target face image;

   Based on the pure white blurred image of the original face image, image decomposition is performed on the first original face blurred image to obtain a second original face blurred image, and a pure white blurred image based on the target face image , performing image decomposition on the first target face blurred image to obtain a second target face blurred image;

   Based on the second original face blurred image, image decomposition is performed on the original face image to obtain an original face detail image; and

   The original face detail image is fused to the second target face blur image to obtain a face fusion image of the target face image.
2. The face fusion method according to claim 1, wherein the first original face blurred image is decomposed based on the pure white blurred image of the original face image to obtain the second original face blurred image Figures, including:

   Normalizing the pixel value of each pixel in the pure white blurred image of the original face image to obtain a first normalized blurred image;

   Divide the pixel value of each pixel in the first original face blur map by the pixel value of each pixel in the first normalized blur map to obtain the second original face blur map.
3. The face fusion method according to claim 2, wherein the pixel value of each pixel in the pure white blur image of the original face image is normalized to obtain a first normalized blur image, include:

   Divide the pixel value of each pixel in the pure white fuzzy image of the original face image by the pure white pixel value to obtain the first normalized fuzzy image; the pure white pixel value is the pixel value of the pure white pixel .
4. The face fusion method according to claim 1, wherein, based on the pure white blurred image of the target face image, image decomposition is performed on the first target face blurred image to obtain a second target face blurred image, include:

   Normalizing the pixel value of each pixel in the pure white blur image of the target face image to obtain a second normalized blur image;

   Divide the pixel value of each pixel point in the first target face blur map by the pixel value of each pixel point in the second normalized blur map to obtain the second target face blur map.
5. The face fusion method according to claim 4, wherein the pixel value of each pixel in the pure white blur image of the target face image is normalized to obtain a second normalized blur image, comprising:

   The second normalized blur image is obtained by dividing the pixel value of each pixel in the pure white blur image of the target face image by the pure white pixel value.
6. The human face fusion method according to claim 1, wherein the image decomposition is performed on the original human face image based on the second original human face blurred image to obtain an original human face detail image, comprising:

   dividing the pixel value of each pixel in the original face image by the pixel value of each pixel in the second original face blur image to obtain the original face detail image;

   The fusion of the original face detail image to the second target face blur image to obtain a face fusion image of the target face image includes:

   Multiply the pixel value of each pixel in the original face detail map by the pixel value of each pixel in the second target face blur map to obtain a face fusion map of the target face map.
7. The face fusion method according to claim 1, wherein the image decomposition is performed on the original face image based on the second original face blurred image to obtain an original face detail image, comprising:

   Subtracting the pixel value of each pixel in the original face image from the pixel value of each pixel in the second original face blur image to obtain the original face detail image;

   The fusion of the original face detail image to the second target face blur image to obtain a face fusion image of the target face image includes:

   The pixel value of each pixel point in the original face detail map is added to the pixel value of each pixel point in the second target face blur map to obtain a face fusion map of the target face map.
8. A face fusion device, comprising:

   an acquisition unit, configured to acquire the original face map and the target face map;

   a face blurring unit, configured to blur the original face image and the target face image respectively to obtain a first original face blurred image and a first target face blurred image;

   The pure white blurring unit is configured to perform a blurring process on the pure white image of the human face generated according to the original human face image to obtain a pure white blurred image of the original human face image, and, The pure white image of the face generated by the image is subjected to fuzzy processing to obtain the pure white blurred image of the target face image;

   a blurring processing unit, configured to perform image decomposition on the first original face blurred image based on the pure white blurred image of the original face image to obtain a second original face blurred image, and, based on the target person The pure white blurred image of the face image, the first target face blurred image is decomposed to obtain the second target face blurred image;

   a detail decomposition unit, configured to perform image decomposition on the original face map based on the second original face blur map to obtain an original face detail map; and

   The face fusion unit is configured to fuse the original face detail image to the second target face fuzzy image to obtain a face fusion image of the target face image.
9. The face fusion device according to claim 8, wherein the blurring processing unit is configured to:

   Normalizing the pixel value of each pixel in the pure white blurred image of the original face image to obtain a first normalized blurred image;

   Divide the pixel value of each pixel in the first original face blur map by the pixel value of each pixel in the first normalized blur map to obtain the second original face blur map.
10. The face fusion device according to claim 9, wherein the blurring processing unit is further configured to:

    Divide the pixel value of each pixel in the pure white fuzzy image of the original face image by the pure white pixel value to obtain the first normalized fuzzy image; the pure white pixel value is the pixel value of the pure white pixel .
11. The human face fusion device according to claim 8, the blurring processing unit is configured to:

    Normalizing the pixel value of each pixel in the pure white blur image of the target face image to obtain a second normalized blur image;

    Divide the pixel value of each pixel point in the first target face blur map by the pixel value of each pixel point in the second normalized blur map to obtain the second target face blur map.
12. The face fusion device according to claim 11, the blurring processing unit is further configured to:

    The second normalized blur image is obtained by dividing the pixel value of each pixel in the pure white blur image of the target face image by the pure white pixel value.
13. The human face fusion device according to claim 8, wherein the detail decomposition unit is configured to:

    dividing the pixel value of each pixel in the original face image by the pixel value of each pixel in the second original face blur image to obtain the original face detail image;

    The face fusion unit is configured as:

    Multiply the pixel value of each pixel in the original face detail map by the pixel value of each pixel in the second target face blur map to obtain a face fusion map of the target face map.
14. The human face fusion device according to claim 8, wherein the detail decomposition unit is configured to:

    Subtracting the pixel value of each pixel in the original face image from the pixel value of each pixel in the second original face blur image to obtain the original face detail image;

    The face fusion unit is configured as:

    The pixel value of each pixel point in the original face detail map is added to the pixel value of each pixel point in the second target face blur map to obtain a face fusion map of the target face map.
15. An electronic device comprising:

    processor;

    a memory for storing the processor-executable instructions;

    wherein the processor is configured to execute the instructions to implement the following steps:

    Obtain the original face map and the target face map;

    The original face image and the target face image are respectively subjected to a blurring process to obtain a first original face blurred image and a first target face blurred image;

    Performing a blurring process on the pure white face image generated according to the original face image, to obtain a pure white blurred image of the original face image, and performing a blurring process on the pure white face image generated according to the target face image The image is blurred to obtain a pure white blurred image of the target face image;

    Based on the pure white blurred image of the original face image, image decomposition is performed on the first original face blurred image to obtain a second original face blurred image, and a pure white blurred image based on the target face image , performing image decomposition on the first target face blurred image to obtain a second target face blurred image;

    Based on the second original face blurred image, image decomposition is performed on the original face image to obtain an original face detail image; and

    The original face detail image is fused to the second target face blur image to obtain a face fusion image of the target face image.
16. A computer-readable storage medium, when the instructions in the computer-readable storage medium are executed by a processor of an electronic device, enable the electronic device to perform the following steps:

    Obtain the original face map and the target face map;

    The original face image and the target face image are respectively subjected to a blurring process to obtain a first original face blurred image and a first target face blurred image;

    Performing a blurring process on the pure white face image generated according to the original face image, to obtain a pure white blurred image of the original face image, and performing a blurring process on the pure white face image generated according to the target face image The image is blurred to obtain a pure white blurred image of the target face image;

    Based on the pure white blurred image of the original face image, image decomposition is performed on the first original face blurred image to obtain a second original face blurred image, and a pure white blurred image based on the target face image , performing image decomposition on the first target face blurred image to obtain a second target face blurred image;

    Based on the second original face blurred image, image decomposition is performed on the original face image to obtain an original face detail image; and

    The original face detail image is fused to the second target face blur image to obtain a face fusion image of the target face image.
17. A computer program product comprising a computer program, wherein the computer program implements the following steps when executed by a processor:

    Obtain the original face map and the target face map;

    The original face image and the target face image are respectively subjected to a blurring process to obtain a first original face blurred image and a first target face blurred image;

    Performing a blurring process on the pure white face image generated according to the original face image, to obtain a pure white blurred image of the original face image, and performing a blurring process on the pure white face image generated according to the target face image The image is blurred to obtain a pure white blurred image of the target face image;

    Based on the pure white blurred image of the original face image, image decomposition is performed on the first original face blurred image to obtain a second original face blurred image, and a pure white blurred image based on the target face image , performing image decomposition on the first target face blurred image to obtain a second target face blurred image;

    Based on the second original face blurred image, image decomposition is performed on the original face image to obtain an original face detail image; and

    The original face detail image is fused to the second target face blur image to obtain a face fusion image of the target face image.

Images