WO2021043121A1 - 一种图像换脸的方法、装置、系统、设备和存储介质 - Google Patents

一种图像换脸的方法、装置、系统、设备和存储介质 Download PDF

Info

Publication number
WO2021043121A1
WO2021043121A1 PCT/CN2020/112777 CN2020112777W WO2021043121A1 WO 2021043121 A1 WO2021043121 A1 WO 2021043121A1 CN 2020112777 W CN2020112777 W CN 2020112777W WO 2021043121 A1 WO2021043121 A1 WO 2021043121A1
Authority
WO
WIPO (PCT)
Prior art keywords
face
changing
person
image
model
Prior art date
Application number
PCT/CN2020/112777
Other languages
English (en)
French (fr)
Inventor
王云
尹淳骥
杨城
Original Assignee
广州华多网络科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 广州华多网络科技有限公司 filed Critical 广州华多网络科技有限公司
Publication of WO2021043121A1 publication Critical patent/WO2021043121A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/04—Context-preserving transformations, e.g. by using an importance map
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/168—Feature extraction; Face representation

Definitions

  • the present invention relates to the field of computer technology, and in particular to a method, device, system, equipment and storage medium for changing faces with images.
  • the video can be a live video or a recorded video.
  • many anchors began to try some new interactive methods, such as changing faces in the video, that is, replacing their own facial images in the video with other people's facial images, and then uploading them to the server for playback.
  • the face-changing solution in the related technology is to use image recognition technology to recognize facial images in video images, then use the facial images of the target person to replace the facial images in the video images, and then upload the face-changing video images to the server.
  • the embodiments of the present application provide a method, device, system, equipment, and storage medium for changing face of an image, which can make the image obtained after face changing processing of the image more realistic.
  • the technical solution is as follows:
  • a method for changing faces in an image is provided.
  • the method is used in a server, and the method includes:
  • model training is performed to obtain a trained first face-changing model and a trained second face-changing model, wherein the trained face
  • the first face-changing model is used to change the face image of the first person to the face image of the second person
  • the second face-changing model after training is used to change the face image of the second person Change the face to the facial image of the first person
  • a method for changing faces with images is provided, the method is used in a terminal, and the method includes:
  • the first image to be changed-face is input to the trained first face-changing model to obtain a second face-changing image.
  • a device for changing faces with images is provided.
  • the device is applied to a server, and the device includes:
  • the receiving module receives the first face change request corresponding to the second account sent by the first terminal logged in by the first account, wherein the face change request carries a set of facial images of the first person;
  • the training module performs model training based on the facial image collection of the first person and the facial image collection of the second person to obtain a first face-changing model after training and a second face-changing model after training, where all
  • the trained first face-changing model is used to change the facial image of the first person to the facial image of the second person
  • the trained second face-changing model is used to change the face of the second person
  • the facial image of is changed to the facial image of the first person
  • the sending module sends the trained first face-changing model to the first terminal, and sends the trained second face-changing model to the second terminal.
  • a device for changing faces with images is provided.
  • the device is applied to a terminal, and the device includes:
  • the sending module sends a face change request corresponding to the second account to the server, wherein the face change request carries a facial image collection of the first person;
  • the receiving module receives the trained first face-changing model sent by the server, wherein the trained first face-changing model is used to change the face image of the first person to that of the second person Facial image
  • the face-changing module when receiving a face-changing instruction corresponding to the second account, inputs the first image to be changed into the trained first face-changing model to obtain a second image after the face-changing.
  • a system for changing faces with images includes a first terminal, a second terminal, and a server, wherein:
  • the server receives a first face change request corresponding to a second account sent by the first terminal logged in with the first account, wherein the face change request carries a set of facial images of the first person; receiving the second account The second face-changing request corresponding to the first account sent by the logged-in second terminal, wherein the face-changing request carries a facial image set of the second person; based on the facial image set of the first person and the The facial image collection of the second person is trained to obtain a first face-changing model after training and a second face-changing model after training, wherein the first face-changing model after training is used to use the first face-changing model after training.
  • the face image of a person is changed to the face image of the second person, and the trained second face changing model is used to change the face image of the second person to the face image of the first person; Sending the trained first face-changing model to the first terminal, and sending the trained second face-changing model to the second terminal;
  • the first terminal sends a face-changing request corresponding to the second account to the server; receiving the trained first face-changing model sent by the server; when receiving a face-changing instruction corresponding to the second account, Input the first image to be changed into the first face-changing model after training to obtain the second image after the face-changing;
  • the second terminal sends a face change request corresponding to the first account to the server; receives the trained second face change model sent by the server; when a face change instruction corresponding to the first account is received, The second image to be changed face is input into the trained second face change model to obtain the first image after face change.
  • a computer device in another aspect, includes one or more processors and one or more memories, and at least one instruction is stored in the one or more memories. Or multiple processors are loaded and executed to implement the operations performed by the image-changing method.
  • a computer-readable storage medium is provided, and at least one instruction is stored in the computer-readable storage medium, and the instruction is loaded and executed by a processor to implement the operations performed by the image-changing method.
  • the embodiment of the present application generates a face-changing model for face-changing processing. After the first image is input into the face-changing model, the second image after the face-changing can be obtained without using the pre-stored facial image of the original person to the facial image of the target person. Partial image replacement is performed, so it does not involve deforming the facial image, so that image distortion can be reduced.
  • FIG. 1 is a schematic diagram of an implementation environment provided by an embodiment of the present application.
  • Figure 2 is a schematic diagram of an implementation environment provided by an embodiment of the present application.
  • FIG. 3 is a schematic diagram of an implementation environment provided by an embodiment of the present application.
  • FIG. 4 is a schematic diagram of an implementation environment provided by an embodiment of the present application.
  • FIG. 5 is a flowchart of an image face changing method provided by an embodiment of the present application.
  • FIG. 6 is a flowchart of an image face changing method provided by an embodiment of the present application.
  • FIG. 7 is a flowchart of an image face changing method provided by an embodiment of the present application.
  • FIG. 8 is a schematic diagram of a specific implementation provided by an embodiment of the present application.
  • FIG. 9 is a schematic structural diagram of an image face-changing device provided by an embodiment of the present application.
  • FIG. 10 is a schematic structural diagram of an image face-changing device provided by an embodiment of the present application.
  • FIG. 11 is a schematic diagram of a terminal structure provided by an embodiment of the present application.
  • Fig. 12 is a schematic diagram of a server structure provided by an embodiment of the present application.
  • Fig. 1, Fig. 2, Fig. 3, and Fig. 4 are implementation environments of an image face-changing method provided by an embodiment of the present invention.
  • the method for providing image face-changing in the present application can be implemented by a terminal and a server.
  • the terminal can run applications with image recording functions, such as live broadcast applications, short video applications, etc.
  • the terminal can have microphones, cameras, speakers and other components.
  • the terminal has communication functions and can access the Internet.
  • the terminal can be a mobile phone or a tablet. Computers, smart wearable devices, desktop computers, notebook computers, etc.
  • the server can be a background server of the above-mentioned application program, and the server can establish communication with the terminal.
  • the server can be a single server or a server group.
  • the server can be responsible for all the processing required by the server in the following schemes. If it is a server group, the different servers in the server group are respectively. It can be responsible for different processing in the following schemes, and the specific processing distribution can be set by the technicians according to actual needs, and will not be repeated here.
  • the method for changing faces in images provided in the embodiments of the present application can replace facial images in video images with facial images of other people, and this function may be referred to as a face changing function.
  • a live application program is taken as an example for detailed description of the solution, and other situations are similar to this, and will not be repeated.
  • a live broadcast application is installed on the terminal.
  • the live broadcast application can record videos and upload them to the Internet. During the recording process, various special effects can be added to the video, or some filter processing can be performed.
  • the host uses the live broadcast application, he can register an account in the live broadcast application.
  • the created account can follow other accounts or be followed by other accounts.
  • the live broadcast application has multiple pages, such as the live broadcast page, the live broadcast list page, etc.
  • the host’s own character page is shown in Figure 1.
  • the character page has personal information such as his account nickname, and also contains a jump control for the face-changing application page and a jump control for terminating the face-changing application page.
  • the face change application page as shown in Figure 2
  • the user can enter the target account that he wants to change the face, the user can manually fill in the target account, or select the target account in the follow account list or the friend account list.
  • the anchor After filling in or selecting the subject of the face change application, enter the facial image capture page, as shown in Figure 3, the anchor can take facial image capture according to the prompts in the live broadcast application.
  • the two anchors who want to change the face communicate with each other. After both parties agree to change the face, the two anchors can send the collection of facial images taken by each to the server.
  • the server can be based on the faces uploaded by the two anchors.
  • the image collection trains the face-changing model, and then sends them to the host respectively, and the subsequent host can start the face-changing live broadcast.
  • the host can click to open the live broadcast control, as shown in Figure 4, click the face-changing control on the live broadcast page to trigger the face-changing live broadcast. All viewers in the live broadcast room can see the live video of the host after changing his face.
  • Fig. 5 is a flowchart on the server side in a method for changing faces in an image provided by an embodiment of the present application. Referring to Figure 2, the process includes:
  • Step 501 Receive a first face change request corresponding to a second account sent by a first terminal logged in by the first account, where the face change request carries a set of facial images of the first person.
  • Step 502 Receive a second face-changing request corresponding to the first account sent by the second terminal logged in with the second account, where the face-changing request carries a set of facial images of the second person.
  • Step 503 Perform model training based on the facial image set of the first person and the facial image set of the second person to obtain the first face-changing model after training and the second face-changing model after training.
  • the face-changing model is used to change the facial image of the first person to the facial image of the second person
  • the trained second face-changing model is used to change the facial image of the second person to the facial image of the first person.
  • Step 504 Send the trained first face-changing model to the first terminal, and send the trained second face-changing model to the second terminal.
  • FIG. 6 is a flowchart on the terminal side in a method for changing faces in an image provided by an embodiment of the present application. Referring to Figure 3, the process includes:
  • Step 601 Send a face change request corresponding to the second account to the server, where the face change request carries a set of facial images of the first person.
  • Step 602 Receive the trained first face-changing model sent by the server, where the trained first face-changing model is used to change the face image of the first person into the face image of the second person.
  • Step 603 When receiving the face-changing instruction corresponding to the second account, input the first image to be changed into the trained first face-changing model to obtain the second face-changing image.
  • FIG. 7 is a flowchart of interaction between a server and a terminal in a method for changing faces in an image provided by an embodiment of the present application. Referring to Figure 7, this embodiment includes:
  • Step 701 The first terminal sends a first face change request corresponding to the second account to the server.
  • the face change request carries a facial image collection of the first person, and the facial image collection may include multiple facial pictures of the first person. These facial pictures may be obtained by taking photos or multiple extracted from videos. Video frame.
  • the first anchor ie, the first person mentioned above
  • the first anchor wants to perform face-changing live broadcast
  • he can run the live broadcast application on the terminal, log in to the first account in the live broadcast application, and then operate in the live broadcast application to enter the character page , Click the jump control on the face-changing application page to jump to the face-changing application page, and perform a face-changing application on the face-changing application page.
  • the first anchor can enter the account identification of one or more accounts (ie the second account) that want to change the face in the face-changing application page, which can be filled in manually, or can be selected in the following account list or friend account list, in After performing the above operations, click the confirmation control to jump to the facial image capture page for facial image capture.
  • the first anchor can take photos through the terminal, or take a video through the terminal, and combine the photos or video frames in the video into a facial image collection.
  • the terminal may generate a first face-changing request, add the captured facial image set and the account identifier of the second account to the first face-changing request, and then send the first face-changing request to the server.
  • the image adjustment function is turned off, such as turning off the functions such as whitening, skin grinding, and filter.
  • the server After the server receives the first face change request from the first terminal, it can have multiple processing methods.
  • the server may send a notification to the second terminal logged in by the second account according to the account identification of the second account carried in the first face change request, and the notification carries the account identification of the first account, It is used to inform the second account that the first account is currently requesting a face-changing live broadcast with it.
  • the second terminal can display prompt information to the user in the live broadcast application to remind the user that other users request a face-changing live broadcast.
  • the second terminal can send the first account corresponding to the first account to the server.
  • the second face change request, the second face change request carries the facial image collection of the second person, which can be used as a confirmation message for the above notification. Then the server performs subsequent processing.
  • Another possible processing method is that the server does not send any notification to the second terminal, but just waits for the second face change request corresponding to the first account sent by the second terminal to the server, and then performs the second face change request when it receives the second face change request.
  • follow-up processing In this way, users can communicate with each other privately, and then send requests separately to realize face-changing live broadcast, or users may not communicate with each other, that is, send face-changing requests to users who want to change their faces. If It just so happens that two users have sent a face-changing request to change faces with each other respectively, and the face-changing live broadcast can be realized.
  • the terminal may play the guidance information and/or display the guidance information, and during the process of playing the guidance information and/or display the guidance information, take the facial image of the first person set.
  • the guidance information is used to instruct the first character to take different actions.
  • the live broadcast application when the facial image collection is captured, the live broadcast application will instruct the image capturing. For example, a paragraph of text is displayed on the top of the screen to instruct the first character to make different actions, such as nodding, shaking his head, opening mouth, closing mouth, smiling, etc.
  • the terminal can also perform the above instructions by broadcasting voice.
  • Step 702 The second terminal sends a second face change request corresponding to the first account to the server.
  • the face change request carries a set of facial images of the second person.
  • the second terminal anchor opens the live broadcast application, first logs in to the second account, and enters Application, then enter the character page, click the jump link of the face change application page, jump to the face change application page, and apply for the face change on the face change application page.
  • the first anchor can enter the desired change in the face change application page
  • the account identification of one or more accounts of the face ie the second account
  • the page performs facial image capture, and after the facial image capture is completed, the account identification of the first account input/selected by the anchor of the second terminal and the facial image collection of the second person captured by the anchor of the second terminal are sent to the server together.
  • the image adjustment function is turned off, for example, functions such as whitening, skin grinding, and filter are turned off.
  • the terminal may play the guidance information and/or display the guidance information, and take the facial image of the second person during the process of playing the guidance information and/or displaying the guidance information set.
  • the guidance information is used to instruct the second character to take different actions.
  • the live broadcast application when the facial image collection is captured, the live broadcast application will instruct the image capturing. For example, a paragraph of text is displayed on the top of the screen to instruct the first character to make different actions, such as nodding, shaking his head, opening mouth, closing mouth, smiling, etc.
  • the terminal can also perform the above instructions by broadcasting voice.
  • step 703 the server performs model training based on the facial image collection of the first person and the facial image collection of the second person to obtain the trained first face-changing model and the trained second face-changing model.
  • the first face-changing model after training is used to change the face image of the first person to the face image of the second person
  • the second face-changing model after training is used to change the face image of the second person to the first person.
  • the server stores the received first account face change request and the second account face change request, and searches for the corresponding second account according to the account identifier contained in the first account face change request, and detects the second account change Face request, if the second account face change request contains the account identifier of the first account, it is determined that the first account and the second account are paired successfully, and the facial image collections of the two accounts are input into the face change model.
  • model training is performed based on the facial image collection of the first person and the facial image collection of the second person to obtain the trained first face-changing model and the trained second face-changing model, including:
  • Step 1 Obtain the facial images in the facial image collection of the first person and the facial image collection of the second person alternately.
  • Step two each time a facial image of the first person is acquired, the facial image of the first person is distorted to obtain the distorted facial image of the first person, and the distorted facial image of the first person is input into the feature extraction model to obtain The first feature image, the first feature image is input to the second restoration model, and the first output image is obtained.
  • the feature extraction model and the second restoration model are parameterized Update.
  • Step three each time a facial image of a second person is acquired, the facial image of the second person is distorted to obtain the distorted facial image of the second person, and the distorted facial image of the second person is input into the feature extraction model to obtain The second feature image, the second feature image is input to the first restoration model to obtain the second output image, and the feature extraction model and the first restoration model are parameterized based on the currently acquired facial image of the second person and the second output image Update.
  • Step 4 After updating the feature extraction model, the first restoration model, and the second restoration model based on the facial images of all the first characters and the facial images of all the second characters, the feature extraction model and the parameters are updated based on the updated parameter After the first restoration model, the trained first face-changing model is determined, and the trained second face-changing model is determined based on the feature extraction model after the parameter update and the second restoration model after the parameter update.
  • the first person’s facial image and the second person’s facial image are distorted in order to train the second restoration model and the first restoration model. Input the distorted facial image to output the restored facial image, and then use
  • the original facial image is the reference training restoration model, and the feature extraction ability of the feature extraction model is trained at the same time.
  • the face image of the first person after distortion is input into the feature extraction model, and the feature extraction model performs feature extraction on the face image of the first person that has been distorted to obtain the first feature image, and input the first feature image into the second The restoration model.
  • the second restoration model restores the first feature image to obtain the first output image. Based on the currently acquired facial image of the first person and the first output image, update the parameters of the feature extraction model and the second restoration model .
  • the face image of the paired second person is distorted to obtain the distorted face image of the second person, and the distorted face image of the second person is input
  • the feature extraction model, the feature extraction model performs feature extraction on the distorted face image of the second person to obtain the second feature image
  • the second feature image is input to the first restoration model
  • the first restoration model restores the second feature image
  • Obtain the second output image and update the parameters of the feature extraction model and the first restoration model based on the currently acquired facial image of the second person and the second output image.
  • the above training cycle is carried out until all facial image sets are input to the face-changing model, and the cycle is stopped.
  • the trained feature extraction model and the first restoration model are the first face-changing model
  • the trained feature extraction model and the first restoration model are the first face-changing model.
  • the second reduction model is the second face-changing model.
  • Step 704 The server sends the trained first face-changing model to the first terminal, and sends the trained second face-changing model to the second terminal.
  • the face-changing model when the face-changing model training is completed, the face-changing model is stored to generate a storage address, and the server according to the account identifier carried in the face-changing request sent by the first terminal and the second terminal, the storage address of the face-changing model And the message of completion of training is sent to the second terminal.
  • the process of sending the trained first face-changing model to the first terminal is similar and will not be repeated one by one.
  • the trained face-changing model when sending the trained face-changing model to the first terminal, if the first account is currently online, send the trained first face-changing model to the first terminal; if the first account is currently offline, Then the trained face changing model is stored, and when it is detected that the first account is switched to the online state, the trained first face changing model is sent to the first terminal.
  • the server when the first account is currently online, sends the message of the face-changing request and the storage address of the face-changing model to the network storage space dedicated to the first account, and the first terminal checks the first account according to a certain period.
  • the network storage space dedicated to the account performs detection, and when the message that the face change request passes is detected, the first terminal automatically downloads it according to the storage address of the face change model.
  • the server sends the message of the face request to the network storage space dedicated to the first account, and stores the message of the face request and the storage address of the face-changing model in the network storage dedicated to the first account space.
  • the first terminal detects the network storage space dedicated to the first account according to a certain period.
  • the face-changing message is detected, the first terminal automatically according to the storage address of the face-changing model To download.
  • the first terminal after the training is sent to the first terminal. Notification of face model deletion.
  • the host of the second terminal After the face-changing model is downloaded to the first terminal, the host of the second terminal enters the termination application page to fill in or select the host in the watch list as the object of the face-changing termination request. After clicking OK, the second terminal sends The server sends a face-changing termination request.
  • the server When receiving the face change termination request of the second terminal, the server sends the deletion notification of the face change model to the network storage space dedicated to the first account according to the account identifier of the first account stored in the server.
  • the first terminal detects the network storage space dedicated to the first account according to a certain period, and directly deletes the first face-changing model stored in the first terminal when it detects that there is a deletion notification of the face-changing model.
  • Step 705 When a face-changing instruction corresponding to the second account is received, input the first image to be changed into the trained first face-changing model to obtain the second face-changing image.
  • the character of the first terminal clicks on the face-changing effect during the live broadcast, and the selection page of the face-changing model appears, select the face-changing model corresponding to the second account face-changing model Button.
  • the camera continuously acquires the first image of the first person.
  • Step 706 When receiving the face-changing instruction corresponding to the first account, input the second image to be changed into the second face-changing model after training, to obtain the first image after the face-changing.
  • the character of the second terminal clicks on the face-changing effect during the live broadcast, and the selection page of the face-changing model appears.
  • the camera continuously acquires the second image of the second person.
  • the image adjustment function can be turned off, and the second image is input into the trained face-changing model.
  • the face-changing model uses the second person’s
  • the facial image is recognized by the encoder, the facial image information of the second person is replaced with the facial image information of the first person, the changed facial image information is input into the decoder to restore the first image, and the face-changing model continues to output the second image after the face-changing , And then add image adjustment function to the first image to beautify the task image presented.
  • the viewers see the face of the first person.
  • FIG 8 is a schematic diagram of a specific implementation provided by an embodiment of this application.
  • the users of two terminals record material and send an application to the server. After receiving the application, the server matches the application and adds the face-changing task to the task queue.
  • the two terminals need to wait for a few days when the user server performs the above operations.
  • a notification of the completion of the model training is issued to the two interrupt sources, and the two terminals download after receiving the notification
  • the face-changing control appears in the user's live broadcast interface after the download is completed, that is, the face-changing function entrance.
  • the embodiment of the present application generates a face-changing model for face-changing processing. After the first image is input into the face-changing model, the second image after the face-changing can be obtained without using the pre-stored facial image of the original person to the facial image of the target person. Partial image replacement is performed, so it does not involve deforming the facial image, so that image distortion can be reduced.
  • An embodiment of the present application provides an image-changing device.
  • the device may be the server in the foregoing embodiment.
  • the device includes:
  • the receiving module 910 receives a first face change request corresponding to the second account sent by the first terminal logged in with the first account, wherein the face change request carries a set of facial images of the first person; A second face-changing request corresponding to the first account sent by the second terminal, wherein the face-changing request carries a set of facial images of the second person.
  • the training module 920 performs model training based on the facial image set of the first person and the facial image set of the second person to obtain the first trained face changing model and the trained second face changing model, wherein,
  • the trained first face-changing model is used to change the face image of the first person to that of the second person
  • the trained second face-changing model is used to change the face of the second person
  • the facial image of the person is changed to the facial image of the first person.
  • the sending module 930 sends the trained first face-changing model to the first terminal, and sends the trained second face-changing model to the second terminal.
  • model training is performed based on the facial image set of the first person and the facial image set of the second person to obtain the first trained face changing model and the trained second face changing model, the The training module 920 is used to:
  • the facial image of the first person is distorted to obtain the distorted facial image of the first person, and the distorted facial image of the first person is input into the feature extraction model to obtain the first person’s facial image.
  • a feature image, the first feature image is input to a second restoration model to obtain a first output image, and based on the currently acquired facial image of the first person and the first output image, the feature extraction model and the The second restores the model and updates the parameters.
  • the facial image of the second person is distorted to obtain the distorted facial image of the second person, and the distorted facial image of the second person is input into the feature extraction model to obtain the first person’s facial image.
  • Two feature images The second feature image is input to the first restoration model to obtain a second output image. Based on the currently acquired facial image of the second person and the second output image, the feature extraction model and the First restore the model and update the parameters.
  • the first restoration model After updating the parameters of the feature extraction model, the first restoration model, and the second restoration model based on the facial images of all the first characters and the facial images of all the second characters, based on the updated feature extraction model
  • the first restoration model after the parameter update determine the first face-changing model after training, and determine the second face-changing model after training based on the feature extraction model after the parameter update and the second restoration model after the parameter update.
  • the sending the trained first face-changing model to the first terminal, and the sending module 930 is used to:
  • the apparatus further includes:
  • the deletion module when receiving a face change termination request corresponding to the first account sent by the second terminal, sends a deletion notification corresponding to the trained first face change model to the first terminal.
  • An embodiment of the present application provides an image-changing device.
  • the device may be the terminal in the foregoing embodiment.
  • the device includes:
  • the sending module 1010 sends a face change request corresponding to the second account to the server, wherein the face change request carries a set of facial images of the first person.
  • the receiving module 1020 receives the trained first face-changing model sent by the server, wherein the trained first face-changing model is used to change the face image of the first person to the second person Face image.
  • the face-changing module 1030 when receiving a face-changing instruction corresponding to the second account, inputs the first image to be changed into the trained first face-changing model to obtain a second image after the face-changing.
  • the apparatus before sending the face change request corresponding to the second account to the server, the apparatus further includes:
  • the guidance module plays guidance information and/or displays guidance information, where the guidance information is used to instruct the first character to take different actions.
  • the photographing module photographs the facial image collection of the first person during the process of playing the guidance information and/or displaying the guidance information.
  • the photographing the facial image collection of the first person, and the photographing module is used to:
  • the embodiment of the present application generates a face-changing model for face-changing processing. After the first image is input into the face-changing model, the second image after the face-changing can be obtained without using the pre-stored facial image of the original person to the facial image of the target person. Partial image replacement is performed, so it does not involve deforming the facial image, so that image distortion can be reduced.
  • the image face-changing device provided in the above embodiment performs image face-changing
  • only the division of the above-mentioned functional modules is used as an example for illustration.
  • the above-mentioned functions can be assigned to different functions according to needs.
  • Module completion that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above.
  • the embodiments of the method for changing faces in an image provided in the above embodiments belong to the same concept. For the specific implementation process, please refer to the method embodiments, which will not be repeated here.
  • An embodiment of the present application also provides a system for changing faces with images.
  • the system includes a first terminal, a second terminal, and a server, wherein:
  • the server receives the first face change request corresponding to the second account sent by the first terminal logged in by the first account, wherein the face change request carries a set of facial images of the first person;
  • the face-changing image is the facial image of the second person, and the trained second face-changing model is used to change the face image of the second person into the facial image of the first person;
  • a terminal sends the trained first face-changing model, and sends the trained second face-changing model to the second terminal.
  • the first terminal sends a face-changing request corresponding to the second account to the server; receiving the trained first face-changing model sent by the server; when receiving a face-changing instruction corresponding to the second account, The first image to be changed face is input into the trained first face changing model to obtain the second image after face changing.
  • the second terminal sends a face change request corresponding to the first account to the server; receives the trained second face change model sent by the server; when a face change instruction corresponding to the first account is received, The second image to be changed face is input into the trained second face change model to obtain the first image after face change.
  • the embodiment of the present application generates a face-changing model for face-changing processing. After the first image is input into the face-changing model, the second image after the face-changing can be obtained without using the pre-stored facial image of the original person to the facial image of the target person. Partial image replacement is performed, so it does not involve deforming the facial image, so that image distortion can be reduced.
  • FIG. 11 is a schematic structural diagram of a terminal provided by an embodiment of the present application.
  • the terminal may be the first terminal or the second terminal in the foregoing embodiment.
  • the terminal 1100 may be: a smart phone, a tablet computer, an MP3 player (Moving Picture Experts Group Audio Layer III, moving picture expert compression standard audio layer 3), MP4 (Moving Picture Experts Group Audio Layer IV, moving picture expert compressing standard audio Level 4) Player, laptop or desktop computer.
  • the terminal 1100 may also be called user equipment, portable terminal, laptop terminal, desktop terminal and other names.
  • the terminal 1100 includes: one or more processors 1101 and one or more memories 1102.
  • the processor 1101 may include one or more processing cores, such as a 4-core processor, an 8-core processor, and so on.
  • the processor 1101 can adopt at least one hardware form among DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array, Programmable Logic Array). achieve.
  • the processor 1101 may also include a main processor and a coprocessor.
  • the main processor is a processor used to process data in the wake state, also called a CPU (Central Processing Unit, central processing unit); the coprocessor is A low-power processor used to process data in the standby state.
  • the processor 1101 may be integrated with a GPU (Graphics Processing Unit, image processor), and the GPU is used to render and draw content that needs to be displayed on the display screen.
  • the processor 1101 may further include an AI (Artificial Intelligence) processor, and the AI processor is used to process computing operations related to machine learning.
  • AI Artificial Intelligence
  • the memory 1102 may include one or more computer-readable storage media, which may be non-transitory.
  • the memory 1102 may also include high-speed random access memory and non-volatile memory, such as one or more magnetic disk storage devices and flash memory storage devices.
  • the non-transitory computer-readable storage medium in the memory 1102 is used to store at least one instruction, and the at least one instruction is used to be executed by the processor 1101 to implement the image conversion provided by the method embodiment of the present application. Face method.
  • the terminal 1100 optionally further includes: a peripheral device interface 1103 and at least one peripheral device.
  • the processor 1101, the memory 1102, and the peripheral device interface 1103 may be connected by a bus or a signal line.
  • Each peripheral device can be connected to the peripheral device interface 1103 through a bus, a signal line, or a circuit board.
  • the peripheral device includes: at least one of a radio frequency circuit 1104, a display screen 1105, a camera 1106, an audio circuit 1107, a positioning component 1108, and a power supply 1109.
  • the peripheral device interface 1103 may be used to connect at least one peripheral device related to I/O (Input/Output) to the processor 1101 and the memory 1102.
  • the processor 1101, the memory 1102, and the peripheral device interface 1103 are integrated on the same chip or circuit board; in some other embodiments, any one of the processor 1101, the memory 1102, and the peripheral device interface 1103 or The two can be implemented on a separate chip or circuit board, which is not limited in this embodiment.
  • the radio frequency circuit 1104 is used for receiving and transmitting RF (Radio Frequency, radio frequency) signals, also called electromagnetic signals.
  • the radio frequency circuit 1104 communicates with a communication network and other communication devices through electromagnetic signals.
  • the radio frequency circuit 1104 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals into electrical signals.
  • the radio frequency circuit 1104 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, and so on.
  • the radio frequency circuit 1104 can communicate with other terminals through at least one wireless communication protocol.
  • the wireless communication protocol includes, but is not limited to: metropolitan area networks, various generations of mobile communication networks (2G, 3G, 4G, and 5G), wireless local area networks, and/or WiFi (Wireless Fidelity, wireless fidelity) networks.
  • the radio frequency circuit 1104 may also include a circuit related to NFC (Near Field Communication), which is not limited in this application.
  • the display screen 1105 is used to display a UI (User Interface, user interface).
  • the UI can include graphics, text, icons, videos, and any combination thereof.
  • the display screen 1105 also has the ability to collect touch signals on or above the surface of the display screen 1105.
  • the touch signal may be input to the processor 1101 as a control signal for processing.
  • the display screen 1105 may also be used to provide virtual buttons and/or virtual keyboards, also called soft buttons and/or soft keyboards.
  • the display screen 1105 there may be one display screen 1105, which is provided with the front panel of the terminal 1100; in other embodiments, there may be at least two display screens 1105, which are respectively arranged on different surfaces of the terminal 1100 or in a folded design; In still other embodiments, the display screen 1105 may be a flexible display screen, which is arranged on a curved surface or a folding surface of the terminal 1100. Furthermore, the display screen 1105 can also be set as a non-rectangular irregular pattern, that is, a special-shaped screen.
  • the display screen 1105 may be made of materials such as LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode, organic light-emitting diode).
  • the camera assembly 1106 is used to capture images or videos.
  • the camera assembly 1106 includes a front camera and a rear camera.
  • the front camera is set on the front panel of the terminal, and the rear camera is set on the back of the terminal.
  • the camera assembly 1106 may also include a flashlight.
  • the flash can be a single-color flash or a dual-color flash. Dual color temperature flash refers to a combination of warm light flash and cold light flash, which can be used for light compensation under different color temperatures.
  • the speaker When the speaker is a piezoelectric ceramic speaker, it can not only convert the electrical signal into human audible sound waves, but also convert the electrical signal into human inaudible sound waves for distance measurement and other purposes.
  • the audio circuit 1107 may also include a headphone jack.
  • the positioning component 1108 is used to locate the current geographic location of the terminal 1100 to implement navigation or LBS (Location Based Service, location-based service).
  • the positioning component 1108 may be a positioning component based on the GPS (Global Positioning System, Global Positioning System) of the United States, the Beidou system of China, the Granus system of Russia, or the Galileo system of the European Union.
  • the power supply 1109 is used to supply power to various components in the terminal 1100.
  • the power source 1109 may be alternating current, direct current, disposable batteries, or rechargeable batteries.
  • the rechargeable battery may support wired charging or wireless charging.
  • the rechargeable battery can also be used to support fast charging technology.
  • the terminal 1100 further includes one or more sensors 1110.
  • the one or more sensors 1110 include, but are not limited to: an acceleration sensor 1111, a gyroscope sensor 1112, a pressure sensor 1113, a fingerprint sensor 1114, an optical sensor 1115, and a proximity sensor 1116.
  • the acceleration sensor 1111 can detect the magnitude of acceleration on the three coordinate axes of the coordinate system established by the terminal 1100.
  • the acceleration sensor 1111 can be used to detect the components of gravitational acceleration on three coordinate axes.
  • the processor 1101 can control the display screen 1105 to display the user interface in a horizontal view or a vertical view according to the gravitational acceleration signal collected by the acceleration sensor 1111.
  • the acceleration sensor 1111 may also be used for the collection of game or user motion data.
  • the gyroscope sensor 1112 can detect the body direction and rotation angle of the terminal 1100, and the gyroscope sensor 1112 can cooperate with the acceleration sensor 1111 to collect the user's 3D actions on the terminal 1100. Based on the data collected by the gyroscope sensor 1112, the processor 1101 can implement the following functions: motion sensing (such as changing the UI according to the user's tilt operation), image stabilization during shooting, game control, and inertial navigation.
  • the pressure sensor 1113 may be arranged on the side frame of the terminal 1100 and/or the lower layer of the display screen 1105.
  • the processor 1101 performs left and right hand recognition or quick operation according to the holding signal collected by the pressure sensor 1113.
  • the processor 1101 controls the operability controls on the UI interface according to the user's pressure operation on the display screen 1105.
  • the operability control includes at least one of a button control, a scroll bar control, an icon control, and a menu control.
  • the fingerprint sensor 1114 is used to collect the user's fingerprint.
  • the processor 1101 can identify the user's identity based on the fingerprint collected by the fingerprint sensor 1114, or the fingerprint sensor 1114 can identify the user's identity based on the collected fingerprints. When it is recognized that the user's identity is a trusted identity, the processor 1101 authorizes the user to perform related sensitive operations, including unlocking the screen, viewing encrypted information, downloading software, paying, and changing settings.
  • the fingerprint sensor 1114 may be provided on the front, back or side of the terminal 1100. When a physical button or a manufacturer logo is provided on the terminal 1100, the fingerprint sensor 1114 may be integrated with the physical button or the manufacturer logo.
  • the optical sensor 1115 is used to collect the ambient light intensity.
  • the processor 1101 may control the display brightness of the display screen 1105 according to the ambient light intensity collected by the optical sensor 1115. Specifically, when the ambient light intensity is high, the display brightness of the display screen 1105 is increased; when the ambient light intensity is low, the display brightness of the display screen 1105 is decreased.
  • the processor 1101 may also dynamically adjust the shooting parameters of the camera assembly 1106 according to the ambient light intensity collected by the optical sensor 1115.
  • the proximity sensor 1116 also called a distance sensor, is usually arranged on the front panel of the terminal 1100.
  • the proximity sensor 1116 is used to collect the distance between the user and the front of the terminal 1100.
  • the processor 1101 controls the display screen 1105 to switch from the on-screen state to the off-screen state; when the proximity sensor 1116 detects When the distance between the user and the front of the terminal 1100 gradually increases, the processor 1101 controls the display screen 1105 to switch from the rest screen state to the bright screen state.
  • FIG. 11 does not constitute a limitation on the terminal 1100, and may include more or fewer components than shown in the figure, or combine certain components, or adopt different component arrangements.
  • FIG. 12 is a schematic structural diagram of a server provided by an embodiment of the present application.
  • the server 1200 may have relatively large differences due to different configurations or performance, and may include one or more processors (central processing units, CPU) 1201 and one Or multiple memories 1202, wherein at least one instruction is stored in the one or more memories 1202, and the at least one instruction is loaded and executed by the one or more processors 1201 to implement the foregoing various method embodiments. Methods.
  • the server 1200 may also have components such as a wired or wireless network interface, a keyboard, an input and output interface for input and output, and the server 1200 may also include other components for implementing device functions, which will not be repeated here.
  • a computer-readable storage medium such as a memory including instructions, which may be executed by a processor to complete the image-changing method in the foregoing embodiment.
  • the computer-readable storage medium may be Read-Only Memory (ROM), Random Access Memory (RAM), Compact Disc Read-Only Memory (CD-ROM), Magnetic tapes, floppy disks and optical data storage devices, etc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Health & Medical Sciences (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Health & Medical Sciences (AREA)
  • Human Computer Interaction (AREA)
  • Multimedia (AREA)
  • Processing Or Creating Images (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

一种图像换脸的方法、装置、系统、设备和存储介质，属于计算机技术领域。所述方法包括：接收第一账户登录的第一终端发送的对应第二账户的第一换脸请求；接收第二账户登录的第二终端发送的对应第一账户的第二换脸请求；基于第一人物的面部图像集合和第二人物的面部图像集合，进行模型训练，得到训练后的第一换脸模型和训练后的第二换脸模型；向第一终端发送训练后的第一换脸模型，向第二终端发送训练后的第二换脸模型。采用本申请，可以减小图像失真。

Description

一种图像换脸的方法、装置、系统、设备和存储介质
本申请要求于2019年09月04日提交中国专利局、申请号为201910833438.0、发明名称为“一种图像换脸的方法、装置、系统、设备和存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本发明涉及计算机技术领域，特别涉及一种图像换脸的方法、装置、系统、设备和存储介质。
背景技术
随着网络技术的飞速发展，网络视频得到了广泛的推广和使用，视频可以是直播视频或录播视频。在录制视频的过程中，很多主播开始尝试一些新的互动方式，例如对视频进行换脸，也即，将视频中自己的面部图像换成其他人的面部图像，然后上传服务器进行播放。
相关技术中的换脸方案是，利用图像识别技术在视频图像中识别面部图像，然后使用目标人物的面部图像，替换视频图像中的面部图像，然后将换脸的视频图像上传服务器。
在实现本申请的过程中，发明人发现现有技术至少存在以下问题：
上述相关技术中对视频图像进行换脸处理的过程中，在视频图像中进行局部图像替换，很可能出现目标人物的面部图像与原人物的面部图像形状不匹配的情况，需要对目标人物的面部图像进行变形处理，这样会导致换脸后的图像存在失真。
发明内容
本申请实施例提供了一种图像换脸的方法、装置、系统、设备和存储介质，能够使图像经换脸处理后得出的图像更加逼真。所述技术方案如下：
一方面，提供了一种图像换脸的方法，所述方法用于服务器，所述方法包括：
接收第一账户登录的第一终端发送的对应第二账户的第一换脸请求， 其中，所述换脸请求中携带有第一人物的面部图像集合；
接收第二账户登录的第二终端发送的对应第一账户的第二换脸请求，其中，所述换脸请求中携带有第二人物的面部图像集合；
基于所述第一人物的面部图像集合和所述第二人物的面部图像集合，进行模型训练，得到训练后的第一换脸模型和训练后的第二换脸模型，其中，所述训练后的第一换脸模型用于将所述第一人物的面部图像换脸为所述第二人物的面部图像，所述训练后的第二换脸模型用于将所述第二人物的面部图像换脸为所述第一人物的面部图像；
向所述第一终端发送所述训练后的第一换脸模型，向所述第二终端发送所述训练后的第二换脸模型。
另一方面，提供了一种图像换脸方法，所述方法用于终端，所述方法包括：
向服务器发送对应第二账户的换脸请求，其中，所述换脸请求中携带有第一人物的面部图像集合；
接收所述服务器发送的训练后的第一换脸模型，其中，所述训练后的第一换脸模型用于将所述第一人物的面部图像换脸为所述第二人物的面部图像；
当接收到对应所述第二账户的换脸指令时，将待换脸的第一图像输入所述训练后的第一换脸模型，得到换脸后的第二图像。
另一方面，提供了一种图像换脸的装置，所述装置应用于服务器，所述装置包括：
接收模块，接收第一账户登录的第一终端发送的对应第二账户的第一换脸请求，其中，所述换脸请求中携带有第一人物的面部图像集合；接收第二账户登录的第二终端发送的对应第一账户的第二换脸请求，其中，所述换脸请求中携带有第二人物的面部图像集合；
训练模块，基于所述第一人物的面部图像集合和所述第二人物的面部图像集合，进行模型训练，得到训练后的第一换脸模型和训练后的第二换脸模型，其中，所述训练后的第一换脸模型用于将所述第一人物的面部图像换脸为所述第二人物的面部图像，所述训练后的第二换脸模型用于将所 述第二人物的面部图像换脸为所述第一人物的面部图像；
发送模块，向所述第一终端发送所述训练后的第一换脸模型，向所述第二终端发送所述训练后的第二换脸模型。
再一方面，提供了一种图像换脸的装置，所述装置应用于终端，所述装置包括：
发送模块，向服务器发送对应第二账户的换脸请求，其中，所述换脸请求中携带有第一人物的面部图像集合；
接收模块，接收所述服务器发送的训练后的第一换脸模型，其中，所述训练后的第一换脸模型用于将所述第一人物的面部图像换脸为所述第二人物的面部图像；
换脸模块，当接收到对应所述第二账户的换脸指令时，将待换脸的第一图像输入所述训练后的第一换脸模型，得到换脸后的第二图像。
再一方面，提供了一种图像换脸的系统，所述系统包括第一终端、第二终端和服务器，其中：
所述服务器，接收第一账户登录的所述第一终端发送的对应第二账户的第一换脸请求，其中，所述换脸请求中携带有第一人物的面部图像集合；接收第二账户登录的所述第二终端发送的对应第一账户的第二换脸请求，其中，所述换脸请求中携带有第二人物的面部图像集合；基于所述第一人物的面部图像集合和所述第二人物的面部图像集合，进行模型训练，得到训练后的第一换脸模型和训练后的第二换脸模型，其中，所述训练后的第一换脸模型用于将所述第一人物的面部图像换脸为所述第二人物的面部图像，所述训练后的第二换脸模型用于将所述第二人物的面部图像换脸为所述第一人物的面部图像；向所述第一终端发送所述训练后的第一换脸模型，向所述第二终端发送所述训练后的第二换脸模型；
所述第一终端，向所述服务器发送对应第二账户的换脸请求；接收所述服务器发送的训练后的第一换脸模型；当接收到对应所述第二账户的换脸指令时，将待换脸的第一图像输入所述训练后的第一换脸模型，得到换脸后的第二图像；
所述第二终端，向所述服务器发送对应第一账户的换脸请求；接收所 述服务器发送的训练后的第二换脸模型；当接收到对应所述第一账户的换脸指令时，将待换脸的第二图像输入所述训练后的第二换脸模型，得到换脸后的第一图像。
再一方面，提供了一种计算机设备，所述计算机设备包括一个或多个处理器和一个或多个存储器，所述一个或多个存储器中存储有至少一条指令，所述指令由所述一个或多个处理器加载并执行以实现所述图像换脸方法所执行的操作。
再一方面，提供了一种计算机可读存储介质，所述计算机可读存储介质中存储有至少一条指令，所述指令由处理器加载并执行以实现所述图像换脸方法所执行的操作。
本申请实施例提供的技术方案带来的有益效果是：
本申请实施例生成换脸模型用于换脸处理，将第一图像输入换脸模型后即可得到换脸后的第二图像，无需使用预存的原人物的面部图像对目标人物的面部图像，进行局部图像替换，所以也不涉及对面部图像进行变形处理，从而，可以减小图像失真。
附图说明
通过阅读下文优选实施方式的详细描述，各种其他的优点和益处对于本领域普通技术人员将变得清楚明了。附图仅用于示出优选实施方式的目的，而并不认为是对本发明的限制。而且在整个附图中，用相同的参考符号表示相同的部件。在附图中：
图1是本申请实施例提供的一种实施环境的示意图；
图2是本申请实施例提供的一种实施环境的示意图；
图3是本申请实施例提供的一种实施环境的示意图；
图4是本申请实施例提供的一种实施环境的示意图；
图5是本申请实施例提供的一种图像换脸方法流程图；
图6是本申请实施例提供的一种图像换脸方法流程图；
图7是本申请实施例提供的一种图像换脸方法流程图；
图8是本申请实施例提供的一种具体实施的示意图；
图9是本申请实施例提供的一种图像换脸装置结构示意图；
图10是本申请实施例提供的一种图像换脸装置结构示意图；
图11是本申请实施例提供的终端结构示意图；
图12是本申请实施例提供的服务器结构示意图。
具体实施方式
下面将参照附图更详细地描述本公开的示例性实施例。虽然附图中显示了本公开的示例性实施例，然而应当理解，可以以各种形式实现本公开而不应被这里阐述的实施例所限制。相反，提供这些实施例是为了能够更透彻地理解本公开，并且能够将本公开的范围完整的传达给本领域的技术人员。
图1、图2、图3、图4是本发明实施例提供的一种图像换脸方法的实施环境，本申请提供图像换脸的方法可以由终端和服务器共同实现。终端可以运行有具有图像录制功能的应用程序，如直播应用程序、短视频应用程序等，终端可以具备麦克风、摄像头、扬声器等部件，终端具有通信功能，可以接入互联网，终端可以是手机、平板电脑、智能穿戴设备、台式计算机、笔记本电脑等。服务器可以是上述应用程序的后台服务器，服务器可以与终端建立通信。该服务器可以是一个单独的服务器也可以是一个服务器组，如果是单独的服务器，该服务器可以负责下述方案中的所有需要由服务器进行的处理，如果是服务器组，服务器组中的不同服务器分别可以负责下述方案中的不同处理，具体的处理分配情况可以由技术人员根据实际需求任意设置，此处不再赘述。
本申请实施例中提供的图像换脸的方法，可以将视频图像中的面部图像替换为其他人物的面部图像，该功能可以称为换脸功能。本申请实施例中以直播应用程序为例进行方案的详细说明，其它情况与之类似，不再赘述。终端上安装有直播应用程序。直播应用程序可以录制视频并将视频上传至网络，在录制的过程中，还可以在视频中添加各种各样的特效，或者进行一些滤镜处理。
主播使用直播应用程序时，可以在直播应用程序中注册一个账户，所创建的账户可以关注其他账户，也可以被其他账户所关注，直播应用程序具有多个页面，如直播页面、直播列表页面等，主播自己的人物页面如图1 所示，在人物页面有自己的账户昵称等个人信息，同时还包含一个换脸申请页面的跳转控件以及一个终止换脸申请页面的跳转控件。在换脸申请页面中，如图2所示，用户可以输入想要换脸的目标账户，用户可以手动填写目标账户也可以在关注账户列表或好友账户列表中选择目标账户。填写或者选择换脸申请的对象后，进入面部图像拍摄页面，如图3所示，主播可根据直播应用程序中的提示进行面部图像拍摄。在进行换脸直播之前，两个想换脸的主播之间进行沟通，双方都同意换脸后，两个主播可以将各自拍摄的面部图像集合发送至服务器，服务器可以基于两个主播上传的面部图像集合训练换脸模型，然后分别发送给主播，后续主播便可以开启换脸直播。
在主播想要进行换脸直播时，主播可以点击开启直播控件，如图4所示，在直播页面中点击换脸控件以触发换脸直播。直播间中的各观众就可以看到该主播换脸后的直播视频。
图5是本申请实施例提供的一种图像换脸的方法中服务器侧的流程图。参见图2，该流程包括：
步骤501，接收第一账户登录的第一终端发送的对应第二账户的第一换脸请求，其中，换脸请求中携带有第一人物的面部图像集合。
步骤502，接收第二账户登录的第二终端发送的对应第一账户的第二换脸请求，其中，换脸请求中携带有第二人物的面部图像集合。
步骤503，基于第一人物的面部图像集合和第二人物的面部图像集合，进行模型训练，得到训练后的第一换脸模型和训练后的第二换脸模型，其中，训练后的第一换脸模型用于将第一人物的面部图像换脸为第二人物的面部图像，训练后的第二换脸模型用于将第二人物的面部图像换脸为第一人物的面部图像。
步骤504，向第一终端发送训练后的第一换脸模型，向第二终端发送训练后的第二换脸模型。
图6是本申请实施例提供的一种图像换脸的方法中终端侧的流程图。参见图3，该流程包括：
步骤601，向服务器发送对应第二账户的换脸请求，其中，换脸请求中 携带有第一人物的面部图像集合。
步骤602，接收服务器发送的训练后的第一换脸模型，其中，训练后的第一换脸模型用于将第一人物的面部图像换脸为第二人物的面部图像。
步骤603，当接收到对应第二账户的换脸指令时，将待换脸的第一图像输入训练后的第一换脸模型，得到换脸后的第二图像。
图7是本申请实施例提供的一种图像换脸的方法中服务器与终端之间交互的流程图。参见图7，该实施例包括：
步骤701，第一终端向服务器发送对应第二账户的第一换脸请求。
其中，换脸请求中携带有第一人物的面部图像集合，面部图像集合可以包括第一人物的多个面部图片，这些面部图片可以是通过拍照得到的，也可以是从视频中提取的多个视频帧。
在实施中，第一主播(即上述第一人物)想要进行换脸直播时，可以在终端运行直播应用程序，在直播应用程序中登陆第一账户，然后在直播应用程序中操作进入人物页面，点击换脸申请页面的跳转控件，跳转至换脸申请页面，在换脸申请页面进行换脸申请。首先第一主播可以在换脸申请页面中输入想要换脸的一个或多个账户(即第二账户)的账户标识，可以手动填写，也可以在关注账户列表或好友账户列表中选择，在执行完上述操作后点击确认控件，跳转至面部图像拍摄页面进行面部图像拍摄。在面部图像拍摄页面中，第一主播可以通过终端拍摄照片，也可以通过终端拍摄视频，将照片或视频中的视频帧组成面部图像集合。在面部图像拍摄完成后，终端可以生成第一换脸请求，在第一换脸请求中添加拍摄的面部图像集合和第二账户的账户标识，然后将第一换脸请求发送给服务器。在上述第一终端上进行的面部图像集合的拍摄中，关闭图像调整功能，如关闭美白、磨皮、滤镜等功能。
服务器在接收到来自第一终端的第一换脸请求后，可以有多种处理方式。
一种可能的处理方式，服务器可以根据第一换脸请求中的携带的第二账户的账户标识向第二账户所登录的第二终端发送通知，该通知中携带有第一账户的账户标识，用于告知第二账户当前有第一账户正请求与其进行 换脸直播。第二终端接收到通知后可以在直播应用程序中向用户显示提示信息，以提示用户有其他用户请求换脸直播，该用户点击确认后，第二终端可以向服务器发送的对应第一账户的第二换脸请求，第二换脸请求携带第二人物的面部图像集合，可以作为对上述通知的确认消息。然后服务器再进行后续处理。
另一种可能的处理方式，服务器不向第二终端发送任何通知，只是等待第二终端向服务器发送的对应第一账户的第二换脸请求，当接收到第二换脸请求时，再进行后续处理。这种方式下，用户之间可以私下沟通好，然后分别发送请求，以实现换脸直播，或者，用户之间也可以不进行沟通，就是分别针对自己想换脸的用户发送换脸请求，如果刚巧有两个用户分别发送了与对方换脸的换脸请求，则可以实现换脸直播。
可选的，向服务器发送对应第二账户的换脸请求之前，终端可以播放指导信息和/或显示指导信息，在播放指导信息和/或显示指导信息的过程中，拍摄第一人物的面部图像集合。
其中，指导信息用于指示第一人物做出不同的动作。
在实施中，进行面部图像集合的拍摄时，直播应用程序会对图像拍摄进行指示。例如，在屏幕上方显示一段文字指示第一人物做出不同的动作，如点头、摇头、张嘴、闭嘴、微笑等，终端还可以通过播报语音的方式进行上述指示。
步骤702，第二终端向服务器发送的对应第一账户的第二换脸请求。
其中，换脸请求中携带有第二人物的面部图像集合。
在实施中，在第二终端向服务器发送对应第一账户的换脸请求之前，如果两换脸主播经协商同意换脸，第二终端的主播打开直播应用程序，首先登陆第二账户，进入到应用程序，然后进入人物页面，点击换脸申请页面跳转链接，跳转至换脸申请页面，在换脸申请页面进行换脸申请，首先第一主播可以在换脸申请页面中输入想要换脸的一个或多个账户(即第二账户)的账户标识，可以手动填写，也可以在关注账户列表或好友账户列表中选择，在执行完上述操作后点击确认控件，跳转至面部图像拍摄页面进行面部图像拍摄，在面部图像拍摄完成后，将第二终端的主播输入/选择的 第一账户的账户标识与第二终端的主播拍摄的第二人物的面部图像集合一并发送到服务器。在上述第二终端上进行的面部图像集合的拍摄中，关闭图像调整功能，如关闭美白、磨皮、滤镜等功能。
可选的，向服务器发送对应第一账户的换脸请求之前，终端可以播放指导信息和/或显示指导信息，在播放指导信息和/或显示指导信息的过程中，拍摄第二人物的面部图像集合。
其中，指导信息用于指示第二人物做出不同的动作。
在实施中，进行面部图像集合的拍摄时，直播应用程序会对图像拍摄进行指示。例如，在屏幕上方显示一段文字指示第一人物做出不同的动作，如点头、摇头、张嘴、闭嘴、微笑等，终端还可以通过播报语音的方式进行上述指示。
步骤703，服务器基于第一人物的面部图像集合和第二人物的面部图像集合，进行模型训练，得到训练后的第一换脸模型和训练后的第二换脸模型。
其中，训练后的第一换脸模型用于将第一人物的面部图像换脸为第二人物的面部图像，训练后的第二换脸模型用于将第二人物的面部图像换脸为第一人物的面部图像。
在实施中，服务器将接收到的第一账户换脸请求和第二账户换脸请求存储，并根据第一账户换脸请求内包含的账户标识寻找对应的第二账户，检测该第二账户换脸请求，如果该第二账户换脸请求中包含第一账户的账户标识，则判定第一账户和第二账户配对成功，将这两个账户的面部图像集合输入换脸模型。
可选的，基于第一人物的面部图像集合和第二人物的面部图像集合，进行模型训练，得到训练后的第一换脸模型和训练后的第二换脸模型，包括：
步骤一，交替获取第一人物的面部图像集合和第二人物的面部图像集合中的面部图像。
步骤二，每获取一个第一人物的面部图像，对第一人物的面部图像进行扭曲，得到扭曲后的第一人物的面部图像，将扭曲后的第一人物的面部 图像输入特征提取模型，得到第一特征图像，将第一特征图像输入第二还原模型，得到第一输出图像，基于当前获取的第一人物的面部图像和第一输出图像，对特征提取模型和第二还原模型，进行参数更新。
步骤三，每获取一个第二人物的面部图像，对第二人物的面部图像进行扭曲，得到扭曲后的第二人物的面部图像，将扭曲后的第二人物的面部图像输入特征提取模型，得到第二特征图像，将第二特征图像输入第一还原模型，得到第二输出图像，基于当前获取的第二人物的面部图像和第二输出图像，对特征提取模型和第一还原模型，进行参数更新。
步骤四，在基于所有第一人物的面部图像和所有第二人物的面部图像对特征提取模型、第一还原模型和第二还原模型进行参数更新后，基于参数更新后的特征提取模型和参数更新后的第一还原模型，确定训练后的第一换脸模型，基于参数更新后的特征提取模型和参数更新后的第二还原模型，确定训练后的第二换脸模型。
其中，将第一人物的面部图像和第二人物的面部图像进行扭曲处理，是为了对第二还原模型和第一还原模型进行训练，通过输入扭曲的面部图像输出还原后的面部图像，再以原面部图像为基准训练还原模型，同时训练特征提取模型的特征提取能力。
在实施中，首先将相同角度、相同表情的第一人物的面部图像和第二人物的面部图像进行配对，然后将配对后的第一人物的面部图像进行扭曲处理，得到扭曲后的第一人物的面部图像，将扭曲后的第一人物的面部图像输入特征提取模型，特征提取模型对该扭曲的第一人物的面部图像进行特征提取，得到第一特征图像，将第一特征图像输入第二还原模型，第二还原模型对第一特征图像进行还原，得到第一输出图像，基于当前获取的第一人物的面部图像和第一输出图像，对特征提取模型和第二还原模型，进行参数更新。
在完成一次特征提取模型和第二还原模型训练后，将配对后的第二人物的面部图像进行扭曲处理，得到扭曲后的第二人物的面部图像，将扭曲后的第二人物的面部图像输入特征提取模型，特征提取模型对该扭曲的第二人物的面部图像进行特征提取，得到第二特征图像，将第二特征图像输 入第一还原模型，第一还原模型对第二特征图像进行还原，得到第二输出图像，基于当前获取的第二人物的面部图像和第二输出图像，对特征提取模型和第一还原模型，进行参数更新。
将上述训练循环进行，直到将所有面部图像集合都输入到换脸模型中，停止该循环，训练后的特征提取模型和第一还原模型是第一换脸模型，训练后的特征提取模型和第二还原模型是第二换脸模型。
步骤704，服务器向第一终端发送训练后的第一换脸模型，向第二终端发送训练后的第二换脸模型。
在实施中，当换脸模型训练完成时，将换脸模型存储，生成存储地址，服务器根据第一终端和第二终端发送的换脸请求中携带的账户标识，将该换脸模型的存储地址和训练完成的消息发送至第二终端。向第一终端发送训练后的第一换脸模型的过程与之类似就不再一一赘述。
可选的，向第一终端发送训练后的换脸模型时，如果第一账户当前处于在线状态，则向第一终端发送训练后的第一换脸模型；如果第一账户当前处于离线状态，则存储训练后的换脸模型，当检测到第一账户切换到在线状态时，向第一终端发送训练后的第一换脸模型。
在实施中，当第一账户当前处于在线状态时，服务器将换脸请求通过的消息及换脸模型的存储地址发送至第一帐户专属的网络存储空间，第一终端按照一定的周期对第一帐户专属的网络存储空间进行检测，当检测到该换脸请求通过的消息时，第一终端自动根据换脸模型的存储地址进行下载。
当第一账户当前处于离线状态时，服务器将脸请求通过的消息发送至第一帐户专属的网络存储空间，将脸请求通过的消息及换脸模型的存储地址存储在第一帐户专属的网络存储空间。当第一账户切换到在线状态时，第一终端按照一定的周期性对第一帐户专属的网络存储空间进行检测，当检测到该换脸消息时，第一终端自动根据换脸模型的存储地址进行下载。
可选的，向第一终端发送训练后的第一换脸模型之后，当接收到第二终端发送的对应第一账户的换脸终止请求时，向第一终端发送对应训练后的第一换脸模型的删除通知。
在实施中，换脸模型下载到第一终端后，第二终端的主播进入终止换脸申请页面填写或者选择在关注列表中的主播作为换脸终止请求的对象，点击确定后，第二终端向服务器发送换脸终止请求。服务器在接收到第二终端的换脸终止请求时，根据服务器内存储的第一账户的账户标识将该换脸模型的删除通知发送至第一帐户专属的网络存储空间。第一终端按照一定的周期对第一帐户专属的网络存储空间进行检测，当检测到有换脸模型的删除通知，直接删除存储在第一终端中的第一换脸模型。
步骤705，当接收到对应第二账户的换脸指令时，将待换脸的第一图像输入训练后的第一换脸模型，得到换脸后的第二图像。
在实施中，第一终端接收到发送的换脸模型后，第一终端的人物在进行直播时，点击换脸效果，出现换脸模型的选择页面，选择对应第二账户换脸模型的换脸按钮。摄像头不断获取第一人物的第一图像，在获取时为了提高换脸处理的准确性，可以关闭图像调整功能，将第一图像输入训练后的换脸模型，首先将第一图像输入特征提取模型，将第一图像中的第一人物的面部图像信息替换成第二人物的面部图像信息，然后将该面部图像信息输入第二还原模型进行还原，得到第二图像，换脸模型持续输出换脸后的第二图像，再对第二图像添加图像调整功能对呈现出的任务形象进行美化。观看直播的观众在观看第一人物的直播时，看到的是第二人物的脸。
步骤706，当接收到对应第一账户的换脸指令时，将待换脸的第二图像输入训练后的第二换脸模型，得到换脸后的第一图像。
在实施中，第二终端接收到发送的换脸模型后，第二终端的人物在进行直播时，点击换脸效果，出现换脸模型的选择页面，选择对应第一账户换脸模型的换脸按钮。摄像头不断获取第二人物的第二图像，在获取时为了提高换脸处理的准确性，可以关闭图像调整功能，将第二图像输入训练后的换脸模型，换脸模型通过将第二人物的面部图像经编码器识别将第二人物的面部图像信息替换成第一人物面部图像信息，将改面部图像信息输入解码器进行还原得到第一图像，换脸模型持续输出换脸后的第二图像，再对第一图像添加图像调整功能对呈现出的任务形象进行美化。观看直播的观众在观看第二人物的直播时，看到的是第一人物的脸。
图8是本申请实施例提供的一种具体实施示意图，两个终端的用户录制素材，向服务器发送申请，服务器接收到申请后对申请进行撮合配对，并将该换脸任务加入任务队列，然后进行模型训练，两个终端在用户服务器进行上述操作时需要等待几天的时间，在训练模型训练完成后，向两个中断源下发模型训练完成的通知，两个终端在接收到通知后下载该训练模型，在下载完成后用户的直播界面中出现该换脸控件，即换脸功能入口。
本申请实施例生成换脸模型用于换脸处理，将第一图像输入换脸模型后即可得到换脸后的第二图像，无需使用预存的原人物的面部图像对目标人物的面部图像，进行局部图像替换，所以也不涉及对面部图像进行变形处理，从而，可以减小图像失真。
本申请实施例提供一种图像换脸的装置，该装置可以是上述实施例中的服务器，如图9所示，所述装置包括：
接收模块910，接收第一账户登录的第一终端发送的对应第二账户的第一换脸请求，其中，所述换脸请求中携带有第一人物的面部图像集合；接收第二账户登录的第二终端发送的对应第一账户的第二换脸请求，其中，所述换脸请求中携带有第二人物的面部图像集合。
训练模块920，基于所述第一人物的面部图像集合和所述第二人物的面部图像集合，进行模型训练，得到训练后的第一换脸模型和训练后的第二换脸模型，其中，所述训练后的第一换脸模型用于将所述第一人物的面部图像换脸为所述第二人物的面部图像，所述训练后的第二换脸模型用于将所述第二人物的面部图像换脸为所述第一人物的面部图像。
发送模块930，向所述第一终端发送所述训练后的第一换脸模型，向所述第二终端发送所述训练后的第二换脸模型。
可选的，基于所述第一人物的面部图像集合和所述第二人物的面部图像集合，进行模型训练，得到训练后的第一换脸模型和训练后的第二换脸模型，所述训练模块920，被用于：
交替获取所述第一人物的面部图像集合和所述第二人物的面部图像集合中的面部图像。
每获取一个第一人物的面部图像，对所述第一人物的面部图像进行扭 曲，得到扭曲后的第一人物的面部图像，将扭曲后的第一人物的面部图像输入特征提取模型，得到第一特征图像，将所述第一特征图像输入第二还原模型，得到第一输出图像，基于当前获取的第一人物的面部图像和所述第一输出图像，对所述特征提取模型和所述第二还原模型，进行参数更新。
每获取一个第二人物的面部图像，对所述第二人物的面部图像进行扭曲，得到扭曲后的第二人物的面部图像，将扭曲后的第二人物的面部图像输入特征提取模型，得到第二特征图像，将所述第二特征图像输入第一还原模型，得到第二输出图像，基于当前获取的第二人物的面部图像和所述第二输出图像，对所述特征提取模型和所述第一还原模型，进行参数更新。
在基于所有第一人物的面部图像和所有第二人物的面部图像对所述特征提取模型、所述第一还原模型和所述第二还原模型进行参数更新后，基于参数更新后的特征提取模型和参数更新后的第一还原模型，确定训练后的第一换脸模型，基于参数更新后的特征提取模型和参数更新后的第二还原模型，确定训练后的第二换脸模型。
可选的，所述向所述第一终端发送所述训练后的第一换脸模型，所述发送模块930，被用于：
如果所述第一账户当前处于在线状态，则向所述第一终端发送所述训练后的第一换脸模型。
如果所述第一账户当前处于离线状态，则存储所述训练后的换脸模型，当检测到所述第一账户切换到在线状态时，向所述第一终端发送所述训练后的第一换脸模型。
如果所述第二账户当前处于在线状态，则向所述第二终端发送所述训练后的第二换脸模型。
如果所述第二账户当前处于离线状态，则存储所述训练后的换脸模型，当检测到所述第二账户切换到在线状态时，向所述第二终端发送所述训练后的第二换脸模型。
可选的，所述向所述第一终端发送所述训练后的第一换脸模型之后，所述装置还包括：
删除模块，当接收到所述第二终端发送的对应所述第一账户的换脸终 止请求时，向所述第一终端发送对应所述训练后的第一换脸模型的删除通知。
本申请实施例提供一种图像换脸的装置，该装置可以是上述实施例中的终端，如图10所示，所述装置包括：
发送模块1010，向服务器发送对应第二账户的换脸请求，其中，所述换脸请求中携带有第一人物的面部图像集合。
接收模块1020，接收所述服务器发送的训练后的第一换脸模型，其中，所述训练后的第一换脸模型用于将所述第一人物的面部图像换脸为所述第二人物的面部图像。
换脸模块1030，当接收到对应所述第二账户的换脸指令时，将待换脸的第一图像输入所述训练后的第一换脸模型，得到换脸后的第二图像。
可选的，所述向服务器发送对应第二账户的换脸请求之前，所述装置还包括：
指导模块，播放指导信息和/或显示指导信息，其中，所述指导信息用于指示所述第一人物做出不同的动作。
拍摄模块，在播放指导信息和/或显示指导信息的过程中，拍摄所述第一人物的面部图像集合。
可选的，所述拍摄所述第一人物的面部图像集合，所述拍摄模块，被用于：
在关闭图像调整功能的状态下，拍摄所述第一人物的面部图像集合。
本申请实施例生成换脸模型用于换脸处理，将第一图像输入换脸模型后即可得到换脸后的第二图像，无需使用预存的原人物的面部图像对目标人物的面部图像，进行局部图像替换，所以也不涉及对面部图像进行变形处理，从而，可以减小图像失真。
需要说明的是：上述实施例提供的图像换脸的装置在进行图像换脸时，仅以上述各功能模块的划分进行举例说明，实际应用中，可以根据需要而将上述功能分配由不同的功能模块完成，即将装置的内部结构划分成不同的功能模块，以完成以上描述的全部或者部分功能。另外，上述实施例提供的图像换脸的方法实施例属于同一构思，其具体实现过程详见方法实施 例，这里不再赘述。
本申请实施例还提供一种图像换脸的系统，所述系统包括第一终端、第二终端和服务器，其中：
所述服务器，接收第一账户登录的第一终端发送的对应第二账户的第一换脸请求，其中，所述换脸请求中携带有第一人物的面部图像集合；接收第二账户登录的第二终端发送的对应第一账户的第二换脸请求，其中，所述换脸请求中携带有第二人物的面部图像集合；基于所述第一人物的面部图像集合和所述第二人物的面部图像集合，进行模型训练，得到训练后的第一换脸模型和训练后的第二换脸模型，其中，所述训练后的第一换脸模型用于将所述第一人物的面部图像换脸为所述第二人物的面部图像，所述训练后的第二换脸模型用于将所述第二人物的面部图像换脸为所述第一人物的面部图像；向所述第一终端发送所述训练后的第一换脸模型，向所述第二终端发送所述训练后的第二换脸模型。
所述第一终端，向所述服务器发送对应第二账户的换脸请求；接收所述服务器发送的训练后的第一换脸模型；当接收到对应所述第二账户的换脸指令时，将待换脸的第一图像输入所述训练后的第一换脸模型，得到换脸后的第二图像。
所述第二终端，向所述服务器发送对应第一账户的换脸请求；接收所述服务器发送的训练后的第二换脸模型；当接收到对应所述第一账户的换脸指令时，将待换脸的第二图像输入所述训练后的第二换脸模型，得到换脸后的第一图像。
本申请实施例生成换脸模型用于换脸处理，将第一图像输入换脸模型后即可得到换脸后的第二图像，无需使用预存的原人物的面部图像对目标人物的面部图像，进行局部图像替换，所以也不涉及对面部图像进行变形处理，从而，可以减小图像失真。
图11是本申请实施例提供的一种终端的结构示意图。该终端可以是上述实施例中的第一终端或第二终端。该终端1100可以是：智能手机、平板电脑、MP3播放器(Moving Picture Experts Group Audio Layer III，动态影像专家压缩标准音频层面3)、MP4(Moving Picture Experts Group Audio Layer  IV，动态影像专家压缩标准音频层面4)播放器、笔记本电脑或台式电脑。终端1100还可能被称为用户设备、便携式终端、膝上型终端、台式终端等其他名称。
通常，终端1100包括有：一个或多个处理器1101和一个或多个存储器1102。
处理器1101可以包括一个或多个处理核心，比如4核心处理器、8核心处理器等。处理器1101可以采用DSP(Digital Signal Processing，数字信号处理)、FPGA(Field－Programmable Gate Array，现场可编程门阵列)、PLA(Programmable Logic Array，可编程逻辑阵列)中的至少一种硬件形式来实现。处理器1101也可以包括主处理器和协处理器，主处理器是用于对在唤醒状态下的数据进行处理的处理器，也称CPU(Central Processing Unit，中央处理器)；协处理器是用于对在待机状态下的数据进行处理的低功耗处理器。在一些实施例中，处理器1101可以在集成有GPU(Graphics Processing Unit，图像处理器)，GPU用于负责显示屏所需要显示的内容的渲染和绘制。一些实施例中，处理器1101还可以包括AI(Artificial Intelligence，人工智能)处理器，该AI处理器用于处理有关机器学习的计算操作。
存储器1102可以包括一个或多个计算机可读存储介质，该计算机可读存储介质可以是非暂态的。存储器1102还可包括高速随机存取存储器，以及非易失性存储器，比如一个或多个磁盘存储设备、闪存存储设备。在一些实施例中，存储器1102中的非暂态的计算机可读存储介质用于存储至少一个指令，该至少一个指令用于被处理器1101所执行以实现本申请中方法实施例提供的图像换脸方法。
在一些实施例中，终端1100还可选包括有：外围设备接口1103和至少一个外围设备。处理器1101、存储器1102和外围设备接口1103之间可以通过总线或信号线相连。各个外围设备可以通过总线、信号线或电路板与外围设备接口1103相连。具体地，外围设备包括：射频电路1104、显示屏1105、摄像头1106、音频电路1107、定位组件1108和电源1109中的至少一种。
外围设备接口1103可被用于将I/O(Input/Output，输入/输出)相关的至少一个外围设备连接到处理器1101和存储器1102。在一些实施例中，处理器 1101、存储器1102和外围设备接口1103被集成在同一芯片或电路板上；在一些其他实施例中，处理器1101、存储器1102和外围设备接口1103中的任意一个或两个可以在单独的芯片或电路板上实现，本实施例对此不加以限定。
射频电路1104用于接收和发射RF(Radio Frequency，射频)信号，也称电磁信号。射频电路1104通过电磁信号与通信网络以及其他通信设备进行通信。射频电路1104将电信号转换为电磁信号进行发送，或者，将接收到的电磁信号转换为电信号。可选地，射频电路1104包括：天线系统、RF收发器、一个或多个放大器、调谐器、振荡器、数字信号处理器、编解码芯片组、用户身份模块卡等等。射频电路1104可以通过至少一种无线通信协议来与其它终端进行通信。该无线通信协议包括但不限于：城域网、各代移动通信网络(2G、3G、4G及5G)、无线局域网和/或WiFi(Wireless Fidelity，无线保真)网络。在一些实施例中，射频电路1104还可以包括NFC(Near Field Communication，近距离无线通信)有关的电路，本申请对此不加以限定。
显示屏1105用于显示UI(User Interface，用户界面)。该UI可以包括图形、文本、图标、视频及其它们的任意组合。当显示屏1105是触摸显示屏时，显示屏1105还具有采集在显示屏1105的表面或表面上方的触摸信号的能力。该触摸信号可以作为控制信号输入至处理器1101进行处理。此时，显示屏1105还可以用于提供虚拟按钮和/或虚拟键盘，也称软按钮和/或软键盘。在一些实施例中，显示屏1105可以为一个，设置终端1100的前面板；在另一些实施例中，显示屏1105可以为至少两个，分别设置在终端1100的不同表面或呈折叠设计；在再一些实施例中，显示屏1105可以是柔性显示屏，设置在终端1100的弯曲表面上或折叠面上。甚至，显示屏1105还可以设置成非矩形的不规则图形，也即异形屏。显示屏1105可以采用LCD(Liquid Crystal Display，液晶显示屏)、OLED(Organic Light-Emitting Diode,有机发光二极管)等材质制备。
摄像头组件1106用于采集图像或视频。可选地，摄像头组件1106包括前置摄像头和后置摄像头。通常，前置摄像头设置在终端的前面板，后置摄像头设置在终端的背面。在一些实施例中，后置摄像头为至少两个，分 别为主摄像头、景深摄像头、广角摄像头、长焦摄像头中的任意一种，以实现主摄像头和景深摄像头融合实现背景虚化功能、主摄像头和广角摄像头融合实现全景拍摄以及VR(Virtual Reality，虚拟现实)拍摄功能或者其它融合拍摄功能。在一些实施例中，摄像头组件1106还可以包括闪光灯。闪光灯可以是单色温闪光灯，也可以是双色温闪光灯。双色温闪光灯是指暖光闪光灯和冷光闪光灯的组合，可以用于不同色温下的光线补偿。
音频电路1107可以包括麦克风和扬声器。麦克风用于采集用户及环境的声波，并将声波转换为电信号输入至处理器1101进行处理，或者输入至射频电路1104以实现语音通信。出于立体声采集或降噪的目的，麦克风可以为多个，分别设置在终端1100的不同部位。麦克风还可以是阵列麦克风或全向采集型麦克风。扬声器则用于将来自处理器1101或射频电路1104的电信号转换为声波。扬声器可以是传统的薄膜扬声器，也可以是压电陶瓷扬声器。当扬声器是压电陶瓷扬声器时，不仅可以将电信号转换为人类可听见的声波，也可以将电信号转换为人类听不见的声波以进行测距等用途。在一些实施例中，音频电路1107还可以包括耳机插孔。
定位组件1108用于定位终端1100的当前地理位置，以实现导航或LBS(Location Based Service，基于位置的服务)。定位组件1108可以是基于美国的GPS(Global Positioning System，全球定位系统)、中国的北斗系统、俄罗斯的格雷纳斯系统或欧盟的伽利略系统的定位组件。
电源1109用于为终端1100中的各个组件进行供电。电源1109可以是交流电、直流电、一次性电池或可充电电池。当电源1109包括可充电电池时，该可充电电池可以支持有线充电或无线充电。该可充电电池还可以用于支持快充技术。
在一些实施例中，终端1100还包括有一个或多个传感器1110。该一个或多个传感器1110包括但不限于：加速度传感器1111、陀螺仪传感器1112、压力传感器1113、指纹传感器1114、光学传感器1115以及接近传感器1116。
加速度传感器1111可以检测以终端1100建立的坐标系的三个坐标轴上的加速度大小。比如，加速度传感器1111可以用于检测重力加速度在三个坐标轴上的分量。处理器1101可以根据加速度传感器1111采集的重力加速 度信号，控制显示屏1105以横向视图或纵向视图进行用户界面的显示。加速度传感器1111还可以用于游戏或者用户的运动数据的采集。
陀螺仪传感器1112可以检测终端1100的机体方向及转动角度，陀螺仪传感器1112可以与加速度传感器1111协同采集用户对终端1100的3D动作。处理器1101根据陀螺仪传感器1112采集的数据，可以实现如下功能：动作感应(比如根据用户的倾斜操作来改变UI)、拍摄时的图像稳定、游戏控制以及惯性导航。
压力传感器1113可以设置在终端1100的侧边框和/或显示屏1105的下层。当压力传感器1113设置在终端1100的侧边框时，可以检测用户对终端1100的握持信号，由处理器1101根据压力传感器1113采集的握持信号进行左右手识别或快捷操作。当压力传感器1113设置在显示屏1105的下层时，由处理器1101根据用户对显示屏1105的压力操作，实现对UI界面上的可操作性控件进行控制。可操作性控件包括按钮控件、滚动条控件、图标控件、菜单控件中的至少一种。
指纹传感器1114用于采集用户的指纹，由处理器1101根据指纹传感器1114采集到的指纹识别用户的身份，或者，由指纹传感器1114根据采集到的指纹识别用户的身份。在识别出用户的身份为可信身份时，由处理器1101授权该用户执行相关的敏感操作，该敏感操作包括解锁屏幕、查看加密信息、下载软件、支付及更改设置等。指纹传感器1114可以被设置终端1100的正面、背面或侧面。当终端1100上设置有物理按键或厂商Logo时，指纹传感器1114可以与物理按键或厂商Logo集成在一起。
光学传感器1115用于采集环境光强度。在一个实施例中，处理器1101可以根据光学传感器1115采集的环境光强度，控制显示屏1105的显示亮度。具体地，当环境光强度较高时，调高显示屏1105的显示亮度；当环境光强度较低时，调低显示屏1105的显示亮度。在另一个实施例中，处理器1101还可以根据光学传感器1115采集的环境光强度，动态调整摄像头组件1106的拍摄参数。
接近传感器1116，也称距离传感器，通常设置在终端1100的前面板。接近传感器1116用于采集用户与终端1100的正面之间的距离。在一个实施 例中，当接近传感器1116检测到用户与终端1100的正面之间的距离逐渐变小时，由处理器1101控制显示屏1105从亮屏状态切换为息屏状态；当接近传感器1116检测到用户与终端1100的正面之间的距离逐渐变大时，由处理器1101控制显示屏1105从息屏状态切换为亮屏状态。
本领域技术人员可以理解，图11中示出的结构并不构成对终端1100的限定，可以包括比图示更多或更少的组件，或者组合某些组件，或者采用不同的组件布置。
图12是本申请实施例提供的一种服务器的结构示意图，该服务器1200可因配置或性能不同而产生比较大的差异，可以包括一个或多个处理器(central processing units，CPU)1201和一个或多个的存储器1202，其中，所述一个或多个存储器1202中存储有至少一条指令，所述至少一条指令由所述一个或多个处理器1201加载并执行以实现上述各个方法实施例提供的方法。当然，该服务器1200还可以具有有线或无线网络接口、键盘以及输入输出接口等部件，以便进行输入输出，该服务器1200还可以包括其他用于实现设备功能的部件，在此不做赘述。
在示例性实施例中，还提供了一种计算机可读存储介质，例如包括指令的存储器，上述指令可由处理器执行以完成上述实施例中的图像换脸方法。例如，该计算机可读存储介质可以是只读存储器(Read-Only Memory,ROM)、随机存取存储器(Random Access Memory，RAM)、只读光盘(Compact Disc Read-Only Memory，CD-ROM)、磁带、软盘和光数据存储设备等。
本领域普通技术人员可以理解实现上述实施例的全部或部分步骤可以通过硬件来完成，也可以通过程序来指令相关的硬件完成，该程序可以存储于一种计算机可读存储介质中，上述提到的存储介质可以是只读存储器，磁盘或光盘等。
上述仅为本申请的较佳实施例，并不用以限制本申请，凡在本申请的精神和原则之内，所作的任何修改、等同替换、改进等，均应包含在本申请的保护范围之内。

Claims (11)

  1. 一种图像换脸的方法，其特征在于，所述方法包括：
    接收第一账户登录的第一终端发送的对应第二账户的第一换脸请求，其中，所述换脸请求中携带有第一人物的面部图像集合；
    接收第二账户登录的第二终端发送的对应第一账户的第二换脸请求，其中，所述换脸请求中携带有第二人物的面部图像集合；
    基于所述第一人物的面部图像集合和所述第二人物的面部图像集合，进行模型训练，得到训练后的第一换脸模型和训练后的第二换脸模型，其中，所述训练后的第一换脸模型用于将所述第一人物的面部图像换脸为所述第二人物的面部图像，所述训练后的第二换脸模型用于将所述第二人物的面部图像换脸为所述第一人物的面部图像；
    向所述第一终端发送所述训练后的第一换脸模型，向所述第二终端发送所述训练后的第二换脸模型。
  2. 根据权利要求1所述的方法，其特征在于，所述基于所述第一人物的面部图像集合和所述第二人物的面部图像集合，进行模型训练，得到训练后的第一换脸模型和训练后的第二换脸模型，包括：
    交替获取所述第一人物的面部图像集合和所述第二人物的面部图像集合中的面部图像；
    每获取一个第一人物的面部图像，对所述第一人物的面部图像进行扭曲，得到扭曲后的第一人物的面部图像，将扭曲后的第一人物的面部图像输入特征提取模型，得到第一特征图像，将所述第一特征图像输入第二还原模型，得到第一输出图像，基于当前获取的第一人物的面部图像和所述第一输出图像，对所述特征提取模型和所述第二还原模型，进行参数更新；
    每获取一个第二人物的面部图像，对所述第二人物的面部图像进行扭曲，得到扭曲后的第二人物的面部图像，将扭曲后的第二人物的面部图像输入特征提取模型，得到第二特征图像，将所述第二特征图像输入第一还原模型，得到第二输出图像，基于当前获取的第二人物的面部图像和所述第二输出图像，对所述特征提取模型和所述第一还原模型，进行参数更新；
    在基于所有第一人物的面部图像和所有第二人物的面部图像对所述特 征提取模型、所述第一还原模型和所述第二还原模型进行参数更新后，基于参数更新后的特征提取模型和参数更新后的第一还原模型，确定训练后的第一换脸模型，基于参数更新后的特征提取模型和参数更新后的第二还原模型，确定训练后的第二换脸模型。
  3. 根据权利要求1所述的方法，其特征在于，所述向所述第一终端发送所述训练后的第一换脸模型之后，所述方法还包括：
    当接收到所述第二终端发送的对应所述第一账户的换脸终止请求时，向所述第一终端发送对应所述训练后的第一换脸模型的删除通知。
  4. 一种图像换脸的方法，其特征在于，所述方法包括：
    向服务器发送对应第二账户的换脸请求，其中，所述换脸请求中携带有第一人物的面部图像集合；
    接收所述服务器发送的训练后的第一换脸模型，其中，所述训练后的第一换脸模型用于将所述第一人物的面部图像换脸为所述第二人物的面部图像；
    当接收到对应所述第二账户的换脸指令时，将待换脸的第一图像输入所述训练后的第一换脸模型，得到换脸后的第二图像。
  5. 根据权利要求4所述的方法，其特征在于，所述向服务器发送对应第二账户的换脸请求之前，所送方法还包括：
    播放指导信息和/或显示指导信息，其中，所述指导信息用于指示所述第一人物做出不同的动作；
    在播放指导信息和/或显示指导信息的过程中，拍摄所述第一人物的面部图像集合。
  6. 根据权利要求5所述的方法，其特征在于，所述拍摄所述第一人物的面部图像集合，包括：
    在关闭图像调整功能的状态下，拍摄所述第一人物的面部图像集合。
  7. 一种图像换脸的装置，其特征在于，所述装置包括：
    接收模块，接收第一账户登录的第一终端发送的对应第二账户的第一换脸请求，其中，所述换脸请求中携带有第一人物的面部图像集合；接收第二账户登录的第二终端发送的对应第一账户的第二换脸请求，其中，所 述换脸请求中携带有第二人物的面部图像集合；
    训练模块，基于所述第一人物的面部图像集合和所述第二人物的面部图像集合，进行模型训练，得到训练后的第一换脸模型和训练后的第二换脸模型，其中，所述训练后的第一换脸模型用于将所述第一人物的面部图像换脸为所述第二人物的面部图像，所述训练后的第二换脸模型用于将所述第二人物的面部图像换脸为所述第一人物的面部图像；
    发送模块，向所述第一终端发送所述训练后的第一换脸模型，向所述第二终端发送所述训练后的第二换脸模型。
  8. 一种图像换脸的装置，其特征在于，所述装置包括：
    发送模块，向所述服务器发送对应第二账户的换脸请求，其中，所述换脸请求中携带有第一人物的面部图像集合；
    接收模块，接收所述服务器发送的训练后的第一换脸模型，其中，所述训练后的第一换脸模型用于将所述第一人物的面部图像换脸为所述第二人物的面部图像；
    换脸模块，当接收到对应所述第二账户的换脸指令时，将待换脸的第一图像输入所述训练后的第一换脸模型，得到换脸后的第二图像。
  9. 一种进行图像换脸的系统，其特征在于，所述系统包括第一终端、第二终端和服务器，其中：
    所述服务器，接收第一账户登录的所述第一终端发送的对应第二账户的第一换脸请求，其中，所述换脸请求中携带有第一人物的面部图像集合；接收第二账户登录的所述第二终端发送的对应第一账户的第二换脸请求，其中，所述换脸请求中携带有第二人物的面部图像集合；基于所述第一人物的面部图像集合和所述第二人物的面部图像集合，进行模型训练，得到训练后的第一换脸模型和训练后的第二换脸模型，其中，所述训练后的第一换脸模型用于将所述第一人物的面部图像换脸为所述第二人物的面部图像，所述训练后的第二换脸模型用于将所述第二人物的面部图像换脸为所述第一人物的面部图像；向所述第一终端发送所述训练后的第一换脸模型，向所述第二终端发送所述训练后的第二换脸模型；
    所述第一终端，向所述服务器发送对应第二账户的换脸请求；接收所 述服务器发送的训练后的第一换脸模型；当接收到对应所述第二账户的换脸指令时，将待换脸的第一图像输入所述训练后的第一换脸模型，得到换脸后的第二图像；
    所述第二终端，向所述服务器发送对应第一账户的换脸请求；接收所述服务器发送的训练后的第二换脸模型；当接收到对应所述第一账户的换脸指令时，将待换脸的第二图像输入所述训练后的第二换脸模型，得到换脸后的第一图像。
  10. 一种计算机设备，其特征在于，所述计算机设备包括一个或多个处理器和一个或多个存储器，所述一个或多个存储器中存储有至少一条指令，所述指令由所述一个或多个处理器加载并执行以实现如权利要求1至权利要求6任一项所述的图像换脸方法所执行的操作。
  11. 一种计算机可读存储介质，其特征在于，所述计算机可读存储介质中存储有至少一条指令，所述指令由处理器加载并执行以实现如权利要求1至权利要求6任一项所述的图像换脸方法所执行的操作。
PCT/CN2020/112777 2019-09-04 2020-09-01 一种图像换脸的方法、装置、系统、设备和存储介质 WO2021043121A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910833438.0A CN110533585B (zh) 2019-09-04 2019-09-04 一种图像换脸的方法、装置、系统、设备和存储介质
CN201910833438.0 2019-09-04

Publications (1)

Publication Number Publication Date
WO2021043121A1 true WO2021043121A1 (zh) 2021-03-11

Family

ID=68666849

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/112777 WO2021043121A1 (zh) 2019-09-04 2020-09-01 一种图像换脸的方法、装置、系统、设备和存储介质

Country Status (2)

Country Link
CN (1) CN110533585B (zh)
WO (1) WO2021043121A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114494002A (zh) * 2022-03-30 2022-05-13 广州公评科技有限公司 一种基于ai换脸视频的原脸图像智能还原方法和系统

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110533585B (zh) * 2019-09-04 2022-09-27 广州方硅信息技术有限公司 一种图像换脸的方法、装置、系统、设备和存储介质
CN111263226B (zh) * 2020-01-17 2021-10-22 中国科学技术大学 视频处理方法、装置、电子设备及介质
CN112752147A (zh) * 2020-09-04 2021-05-04 腾讯科技（深圳）有限公司 视频处理方法、装置和存储介质
CN113487745A (zh) * 2021-07-16 2021-10-08 思享智汇(海南)科技有限责任公司 一种增强现实的方法、装置及系统

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106534757A (zh) * 2016-11-22 2017-03-22 北京金山安全软件有限公司 人脸交换方法、装置、主播终端及观众终端
US9697266B1 (en) * 2013-09-27 2017-07-04 EMC IP Holding Company LLC Management of computing system element migration
CN108040290A (zh) * 2017-12-22 2018-05-15 四川长虹电器股份有限公司 基于ar技术的电视节目实时换脸方法
CN109063658A (zh) * 2018-08-08 2018-12-21 吴培希 一种利用深度学习在多移动终端视频人物换脸的方法
CN110533585A (zh) * 2019-09-04 2019-12-03 广州华多网络科技有限公司 一种图像换脸的方法、装置、系统、设备和存储介质

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9165182B2 (en) * 2013-08-19 2015-10-20 Cisco Technology, Inc. Method and apparatus for using face detection information to improve speaker segmentation
CN104244022B (zh) * 2014-08-29 2018-03-09 形山科技（深圳）有限公司 一种图像处理方法及系统
CN106331569B (zh) * 2016-08-23 2019-08-30 广州华多网络科技有限公司 即时视频画面中人物脸部变换方法以及系统
CN108347578B (zh) * 2017-01-23 2020-05-08 腾讯科技（深圳）有限公司 视频通话中视频图像的处理方法及装置
CN107564080B (zh) * 2017-08-17 2020-07-28 北京觅己科技有限公司 一种人脸图像的替换系统

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9697266B1 (en) * 2013-09-27 2017-07-04 EMC IP Holding Company LLC Management of computing system element migration
CN106534757A (zh) * 2016-11-22 2017-03-22 北京金山安全软件有限公司 人脸交换方法、装置、主播终端及观众终端
CN108040290A (zh) * 2017-12-22 2018-05-15 四川长虹电器股份有限公司 基于ar技术的电视节目实时换脸方法
CN109063658A (zh) * 2018-08-08 2018-12-21 吴培希 一种利用深度学习在多移动终端视频人物换脸的方法
CN110533585A (zh) * 2019-09-04 2019-12-03 广州华多网络科技有限公司 一种图像换脸的方法、装置、系统、设备和存储介质

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114494002A (zh) * 2022-03-30 2022-05-13 广州公评科技有限公司 一种基于ai换脸视频的原脸图像智能还原方法和系统

Also Published As

Publication number Publication date
CN110533585B (zh) 2022-09-27
CN110533585A (zh) 2019-12-03

Similar Documents

Publication Publication Date Title
CN109600678B (zh) 信息展示方法、装置及系统、服务器、终端、存储介质
WO2021043121A1 (zh) 一种图像换脸的方法、装置、系统、设备和存储介质
CN108401124B (zh) 视频录制的方法和装置
CN110572711B (zh) 视频封面生成方法、装置、计算机设备及存储介质
CN109729372B (zh) 直播间切换方法、装置、终端、服务器及存储介质
WO2019114514A1 (zh) 在直播间显示音高信息的方法及装置、存储介质
CN110971930A (zh) 虚拟形象直播的方法、装置、终端及存储介质
WO2019128592A1 (zh) 进行直播的方法和装置
CN110300274B (zh) 视频文件的录制方法、装置及存储介质
CN112118477B (zh) 虚拟礼物展示方法、装置、设备以及存储介质
CN110992493A (zh) 图像处理方法、装置、电子设备及存储介质
CN110324689B (zh) 音视频同步播放的方法、装置、终端及存储介质
CN111246095B (zh) 控制镜头运动的方法、装置、设备及存储介质
CN111355974A (zh) 虚拟礼物赠送处理的方法、装置、系统、设备及存储介质
CN112533017B (zh) 直播方法、装置、终端及存储介质
WO2023000677A1 (zh) 内容项显示方法及装置
CN109982129B (zh) 短视频的播放控制方法、装置及存储介质
WO2022095465A1 (zh) 信息显示方法及装置
CN108848405B (zh) 图像处理方法和装置
CN111028566A (zh) 直播教学的方法、装置、终端和存储介质
CN112788359A (zh) 直播处理方法、装置、电子设备及存储介质
CN111880888A (zh) 预览封面生成方法、装置、电子设备及存储介质
CN111083513B (zh) 直播画面处理方法、装置、终端及计算机可读存储介质
CN111083526B (zh) 视频转场方法、装置、计算机设备及存储介质
CN112468884A (zh) 动态资源显示方法、装置、终端、服务器及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20860115

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20860115

Country of ref document: EP

Kind code of ref document: A1