WO2021180114A1 - Procédé et appareil de reconstruction de visage, dispositif informatique et support de stockage - Google Patents

Procédé et appareil de reconstruction de visage, dispositif informatique et support de stockage Download PDF

Info

Publication number
WO2021180114A1
WO2021180114A1 PCT/CN2021/079934 CN2021079934W WO2021180114A1 WO 2021180114 A1 WO2021180114 A1 WO 2021180114A1 CN 2021079934 W CN2021079934 W CN 2021079934W WO 2021180114 A1 WO2021180114 A1 WO 2021180114A1
Authority
WO
WIPO (PCT)
Prior art keywords
face
reconstructed
video stream
feature set
reconstruction
Prior art date
Application number
PCT/CN2021/079934
Other languages
English (en)
Chinese (zh)
Inventor
王文斓
Original Assignee
广州虎牙科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 广州虎牙科技有限公司 filed Critical 广州虎牙科技有限公司
Publication of WO2021180114A1 publication Critical patent/WO2021180114A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/00—2D [Two Dimensional] image generation
    • G06T11/001—Texturing; Colouring; Generation of texture or colour
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/168—Feature extraction; Face representation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/21—Server components or server architectures
    • H04N21/218—Source of audio or video content, e.g. local disk arrays
    • H04N21/2187—Live feed

Definitions

  • This application relates to the field of image processing technology, for example, to a face reconstruction method, device, computer equipment, and storage medium.
  • the host can generate a live video stream through camera capture or screen recording, and send the live video stream to multiple viewers in the live room through the server.
  • the viewers in the live broadcast room generally can only adjust the image content in the live video stream through simple image overlay methods, such as adding special effects.
  • the above adjustment process is implemented by the server, and once adjusted, it will Multiple audiences take effect at the same time.
  • the live video stream image adjustment method based on image superposition has a single implementation method and a wide range of effects, and it cannot meet people's growing personalized live viewing needs.
  • This application provides a face reconstruction method, device, computer equipment, and storage medium to provide a new way to adjust the image content in a live video stream and enrich the functions of the live broadcast room.
  • a face reconstruction method including:
  • the face reconstruction parameters generate a reconstructed face that matches the face of the anchor
  • a face reconstruction device including:
  • the face reconstruction parameter acquisition module is set to acquire the face reconstruction parameters of the host’s face in the live video stream by the target audience;
  • the reconstructed face generation module is set to generate a reconstructed face that matches the face of the anchor according to the face reconstruction parameters;
  • the replacement display module is set to replace the face of the anchor in the live video stream with a reconstructed face, and then provide it to the target audience for display.
  • a computer device is also provided, and the device includes:
  • One or more processors are One or more processors;
  • Storage device set to store one or more programs
  • the one or more processors implement the face reconstruction method provided by any embodiment of the present application.
  • a computer-readable storage medium which stores a computer program, and when the computer program is executed by a processor, it implements the face reconstruction method provided by any embodiment of the present application.
  • Fig. 1a is a flowchart of a face reconstruction method in Embodiment 1 of the present application.
  • FIG. 1b is a schematic diagram of a face reconstruction parameter setting interface in Embodiment 1 of the present application.
  • FIG. 1c is a flowchart of a face reconstruction process in Embodiment 1 of the present application.
  • FIG. 1d is a schematic structural diagram of a face encoder in Embodiment 1 of the present application.
  • Fig. 2a is a flowchart of a face reconstruction method in the second embodiment of the present application.
  • Fig. 2b is a schematic structural diagram of a face generator in the second embodiment of the present application.
  • Fig. 3 is a schematic structural diagram of a face reconstruction device in the third embodiment of the present application.
  • Fig. 4 is a schematic structural diagram of a computer device in the fourth embodiment of the present application.
  • Fig. 1a is a flowchart of a face reconstruction method in the first embodiment of the present application. This embodiment can be applied to the situation of performing feature adjustment on the face of the anchor in the live video.
  • the method can be used by the face reconstruction device
  • the device can be implemented by hardware and/or software, and generally can be integrated in a device that provides image processing services, for example, an audience device, or an edge node directly connected to the audience device.
  • the method includes the following steps.
  • Step 110 Obtain the face reconstruction parameters of the host's face in the live video stream by the target audience.
  • the target audience can be any audience who is watching the live video.
  • the face reconstruction parameters are set by the target audience to make the host’s face more in line with his own aesthetics. Corresponding adjustments made by the facial features of the face.
  • acquiring the face reconstruction parameters of the target audience on the host’s face in the live video stream may include: responding to the host’s face reconstruction request of the target audience, providing the face reconstruction parameter setting interface to the target audience,
  • the face reconstruction parameter setting interface includes at least one face reconstruction setting item; in response to the target audience’s input setting for at least one face reconstruction setting item, the target audience’s facial recognition of the host’s face in the live video stream is obtained.
  • the face reconstruction parameter setting page includes multiple face reconstruction settings such as face-lifting, hairstyle, masculine, feminine, and aging.
  • face reconstruction setting items such as selecting new hairstyles and hair colors, changing the masculine ratio of the face, and changing the youthful ratio of the face, you can obtain the target audience’s view of the host’s face in the live video stream.
  • Face reconstruction parameters such as hairstyle A, hair color B, masculine 50%, and 30% younger.
  • the target audience may also include: obtaining the mixed data stream sent by the host.
  • the mixed data stream includes: the original video stream and the original video stream.
  • the basic converged video stream is used as a live video stream and provided to the target audience.
  • the content delivery network (Content Delivery Network) Network, CDN) to obtain the original video stream sent by the host and the basic facial feature set corresponding to the host’s face in the original video stream, where the basic facial feature set refers to the real host who has not been adjusted by the target audience
  • the facial feature set of the face is the basis for adjusting at least one facial feature on the anchor’s face.
  • the basic facial feature set is input into the face generator to generate the corresponding anchor basic face, and the anchor basic
  • the human face is fused with the host's face in the original video stream, and the generated basic fused video stream is sent to the target audience as a live video stream.
  • the host while encoding the original video stream, the host will obtain the corresponding face area of the host from the original video stream. By inputting the face area of the host into the face encoder, it can obtain the real The set of basic facial features corresponding to the face of the anchor.
  • the basic face feature set is subjected to lossless compression in the encoding fusion stage, for example, Huffman encoding, which forms a mixed data stream with the encoded original video stream, and distributes it to multiple devices through the CDN network.
  • the face encoder extracts the basic face feature set from the input face picture through a neural network.
  • the structure diagram of the face encoder is shown in Fig. 1d.
  • CNN represents the convolutional layer
  • Fully Connected (FC) represents the fully connected layer
  • x represents the input face picture
  • the extracted basic face feature set which implies the identity of the input face, and attributes such as hairstyle, hair color, expression, gender, and background.
  • Step 120 Generate a reconstructed face that matches the face of the anchor according to the face reconstruction parameters.
  • the face key points and corresponding coordinates of the face of the host can be obtained by image processing on the face image of the host, according to The acquired face reconstruction parameters determine the face key points that need to be changed and the corresponding coordinate changes, and adjust the coordinates of the face key points according to the coordinate changes to obtain a reconstructed face that matches the face of the anchor.
  • the basic face feature set corresponding to the face of the host can also be adjusted to generate a reconstructed face that matches the face of the host.
  • any basic face feature in the basic face feature set corresponds to multiple face key points, for example, the basic face feature "eyebrows" corresponds to multiple face key points on the contour of the eyebrows.
  • generating a reconstructed face that matches the face of the anchor may include: obtaining a basic face feature set corresponding to the face of the anchor in the live video stream; according to the face reconstruction parameters, The basic face feature set is adjusted to generate a reconstructed face feature set; according to the reconstructed face feature set, a reconstructed face that matches the face of the anchor is generated.
  • the essence of adjusting the face of the host is to adjust the basic face feature set corresponding to the face of the host according to the face reconstruction parameters, it is necessary to obtain
  • the basic facial feature set corresponding to the host’s face in the live video stream is based on the facial reconstruction parameters set by the target audience such as hairstyle A, hair color B, face-lifting 10%, expression C, and youth 30%.
  • the face feature set is adjusted accordingly to generate a reconstructed face feature set, and then according to the reconstructed face feature set, a reconstructed face that is preferred by the target audience and matches the face of the anchor is generated.
  • each face reconstruction setting item in the face reconstruction parameter setting interface is associated with one or more face features in the basic face feature set.
  • each face reconstruction setting item can adjust the face of the anchor accordingly, and each face reconstruction setting item can adjust at least one face feature in the basic face feature set.
  • the face reconstruction setting item "30% younger” needs to be modified separately for the anchor's facial hair color, skin color, skin condition and other characteristics; the face reconstruction setting item "hairstyle A" is only for the anchor person The hairstyle of the face is modified.
  • Step 130 After replacing the face of the anchor in the live video stream with the reconstructed face, it is provided to the target audience for display.
  • the target audience for display which may include: fusing the reconstructed face with the face of the host in the live video stream to generate a reconstruction face. Construct a fusion video stream; provide the reconstructed fusion video stream to the target audience for display.
  • the host face in the live video stream is replaced with the reconstructed face that the target audience likes, and the reconstructed fusion video is obtained.
  • Streaming by providing the reconstructed and fused video stream to the target audience for display, it achieves the effect of improving the target audience's satisfaction with the live video and the residence time in the current live broadcast room.
  • the viewer is provided with the function of adjusting the features of the host’s face in the live video, so that the viewer is satisfied with the live content of a host as a whole, but when a display content is unacceptable, he can respond to the host according to his own preferences.
  • the face is adjusted to prevent viewers from frequently switching between live broadcasts within a certain period of time, so as to reduce the pressure on the live broadcast platform and the consumption of memory, and improve the service quality of the live broadcast platform.
  • the face reconstruction parameters of the host’s face in the live video stream are acquired by the target audience; the reconstructed face matching the host’s face is generated according to the face reconstruction parameters; After the host’s face is replaced with a reconstructed face, it is provided to the target audience for display, which solves the problem that the audience can only passively receive the live content and cannot make any adjustments or modifications.
  • the new way of adjusting the image content allows viewers to adjust the face of the anchor in the live video they are watching according to their own preferences, enriching the functions of the live room.
  • the audience device when the device is an audience device, can directly obtain the face reconstruction parameters of the target audience on the host's face in the live video stream, generate a reconstructed face that matches the host's face, and Replace the host’s face in the live video stream with the reconstructed face and display it to the target audience; when the device is an edge node directly connected to the audience device, the edge node obtains the target audience’s live video from the audience device After the face reconstruction parameters of the host’s face in the stream, the face reconstruction parameters are obtained from the audience device, and a reconstructed face matching the host’s face is generated, and the host’s face in the live video stream is replaced In order to reconstruct the face and feed it back to the audience device, the replaced live video stream is displayed to the target audience through the audience device.
  • Fig. 2a is a flowchart of a face reconstruction method in the second embodiment of the present application. This embodiment can be combined with multiple alternative solutions in the foregoing embodiment. In this embodiment, the method of generating the reconstructed face feature set will be described. Referring to Figure 2a, the method may include the following steps.
  • Step 210 Obtain the face reconstruction parameters of the face of the anchor in the live video stream by the target audience.
  • Step 220 Obtain a basic face feature set corresponding to the face of the anchor in the live video stream.
  • Step 230 Adjust the basic face feature set according to the face reconstruction parameters to generate a reconstructed face feature set.
  • the reconstructed face feature set can be generated by replacing the specified face features in the basic face feature set.
  • the specified face features in the face feature set are adjusted to generate the reconstructed face feature set, or the reconstructed face feature set can be generated through the combination of feature replacement and feature adjustment.
  • adjust the basic face feature set according to the face reconstruction parameters to generate the reconstructed face feature set may include: according to the face reconstruction parameters, as well as preset face reconstruction parameters and feature corrections
  • the mapping relationship between the values determines at least one feature correction value; according to at least one feature correction value and the correction direction matching the at least one feature correction value, the basic face feature set is adjusted to generate a reconstructed face Feature set.
  • the face reconstruction parameter is masculine 30% corresponds to the basic face feature D
  • the feature correction value is M.
  • the mapping relationship between the face reconstruction parameters and the feature correction value needs to be set in advance, so that after obtaining the face reconstruction parameters of the target audience, you can search
  • the mapping relationship determines at least one feature correction value, for example, eyes 20%, and then the corresponding basic facial features can be adjusted according to the correction direction matching the feature correction value, for example, the eyes are reduced by 20% to generate a reconstructed person Face feature set.
  • two sets of sample pictures can be selected in advance.
  • feature extraction is performed on the two sets of sample pictures and the feature average value is calculated, and the feature correction value corresponding to the face reconstruction parameter is obtained by comparing the two face feature average values.
  • the preset mapping relationship between the face reconstruction parameters and the feature correction values can be obtained.
  • one sample picture corresponds to the eyes before adjustment
  • the other sample picture corresponds to the eyes adjusted using the face reconstruction parameter J.
  • Decrease the dimension and get ⁇ z_eye ⁇
  • adjusting the basic face feature set according to the face reconstruction parameters to generate the reconstructed face feature set may include: determining at least one replacement face feature according to the face reconstruction parameters; The term replaces the face features, replaces the corresponding basic face features in the basic face feature set, and generates a reconstructed face feature set.
  • the feature replacement method when used to generate the reconstructed face feature set, at least one replacement face feature is determined according to the face reconstruction parameters, the matching basic face feature is found in the basic face feature set, and Substituting the corresponding replacement facial features to obtain the reconstructed facial feature set.
  • the replacement face feature is z 1 '.
  • the feature vector z corresponding to the hair color feature is found in the basic face feature set (z 1 ...z N) 1.
  • Replace z 1 with the feature vector z 1 'corresponding to hair color B to obtain the reconstructed face feature set (z 1 '...z N ).
  • Step 240 According to the reconstructed face feature set, a reconstructed face that matches the face of the anchor is generated.
  • generating a reconstructed face that matches the face of the anchor may include: inputting the reconstructed face feature set into the face generator to obtain a reconstituted face that matches the face of the anchor Construct a face; where the face generator includes multiple connected convolutional layers, and multiple reconstructed face features in the reconstructed face feature set are used as the input of multiple convolutional layers; the reconstructed face features and Facial attribute association.
  • the face generator generates a corresponding reconstructed face image according to the input reconstructed face feature set through a neural network.
  • the structure diagram of the face generator is shown in Figure 2b, where CNN represents the convolutional layer, FC represents the fully connected layer, x'represents the synthesized reconstructed face image, const is any fixed value, z 1 ...z N Represents the N face feature vectors in the reconstructed face feature set, z 1 ... z N are respectively transformed into tensors of corresponding dimensions through their respective fully connected layers, and these tensors are used to modulate the output results of the convolutional layer, for example Instance normalization (instance normalization) and weight demodulation (weight demodulation), etc., and use the result as the input of the next layer of convolutional layer.
  • Instance normalization instance normalization
  • weight demodulation weight demodulation
  • Step 250 After replacing the face of the anchor in the live video stream with the reconstructed face, it is provided to the target audience for display.
  • the face reconstruction parameters of the host’s face in the live video stream are acquired by the target audience; the reconstructed face matching the host’s face is generated according to the face reconstruction parameters; After replacing the host’s face with a reconstructed face, it is provided to the target audience for display, which solves the problem that the audience can only passively receive the live content and cannot make any adjustments or modifications.
  • the viewer is provided with the host’s face in the live video.
  • the feature adjustment function allows viewers to adjust the face of the anchor in the live video they are watching according to their own preferences, so as to maximize the audience's residence time in the same live broadcast room and reduce the viewer's live broadcast room switching rate.
  • Fig. 3 is a schematic structural diagram of a face reconstruction device in the third embodiment of the present application. This embodiment can be applied to the situation of adjusting the features of the host’s face in the live video.
  • the device can be implemented by hardware and/or software. It can be implemented, and can generally be integrated in a device that provides image processing services, such as an audience device, or an edge node directly connected to the audience device.
  • the face reconstruction device includes: a face reconstruction parameter acquisition module 310, a reconstructed face generation module 320, and a replacement display module 330.
  • the face reconstruction parameter acquisition module 310 is configured to acquire the face reconstruction parameters of the host's face in the live video stream by the target audience;
  • the reconstructed face generation module 320 is configured to generate a reconstructed face that matches the face of the anchor according to the face reconstruction parameters;
  • the replacement display module 330 is configured to replace the face of the host in the live video stream with a reconstructed face, and then provide it to the target audience for display.
  • the face reconstruction parameters of the host’s face in the live video stream are acquired by the target audience; the reconstructed face matching the host’s face is generated according to the face reconstruction parameters; After the host’s face is replaced with a reconstructed face, it is provided to the target audience for display, which solves the problem that the audience can only passively receive the live content and cannot make any adjustments or modifications.
  • the new way of adjusting the image content enriches the functions of the live broadcast room.
  • the face reconstruction parameter acquisition module 310 is configured to provide the face reconstruction parameter setting interface to the target audience in response to the host’s face reconstruction request of the target audience, and the face reconstruction parameter setting interface At least one face reconstruction setting item is included; in response to the input setting of the target audience for the at least one face reconstruction setting item, the face reconstruction parameters of the target audience for the host's face in the live video stream are acquired.
  • the reconstructed face generation module 320 is configured to: obtain a basic face feature set corresponding to the face of the anchor in the live video stream; adjust the basic face feature set according to the face reconstruction parameters to generate Reconstruct the face feature set; according to the reconstructed face feature set, generate a reconstructed face that matches the host's face.
  • each face reconstruction setting item in the face reconstruction parameter setting interface is associated with one or more face features in the basic face feature set.
  • the replacement display module 330 is configured to: fuse the reconstructed face with the host face in the live video stream to generate a reconstructed fused video stream; and provide the reconstructed fused video stream to the target audience for display.
  • the face reconstruction parameter acquisition module 310 is further configured to: before acquiring the face reconstruction parameters of the target audience on the host’s face in the live video stream, acquire the mixed data stream sent by the host, where the mixed data
  • the stream includes: the original video stream, and the basic face feature set corresponding to the host's face in the original video stream; based on the basic face feature set in the mixed data stream, the anchor's basic face is generated; the anchor's basic face is combined with the original video
  • the face of the anchor in the stream is fused to generate a basic fusion video stream; the basic fusion video stream is used as a live video stream and provided to the target audience.
  • the reconstructed face generation module 320 is configured to determine at least one feature correction value according to the face reconstruction parameter and the preset mapping relationship between the face reconstruction parameter and the feature correction value; According to at least one feature correction value and a correction direction matching the at least one feature correction value, the basic face feature set is adjusted to generate a reconstructed face feature set.
  • the reconstructed face generation module 320 is configured to: determine at least one replacement face feature according to the face reconstruction parameters; replace the corresponding basis in the basic face feature set according to the at least one replacement face feature Face features, generating a reconstructed face feature set.
  • the reconstructed face generation module 320 is configured to: input the reconstructed face feature set into the face generator to obtain a reconstructed face matching the face of the anchor; where, in the face generator It includes multiple connected convolutional layers, and multiple reconstructed face features in the reconstructed face feature set are respectively used as the input of multiple convolutional layers; the reconstructed face features are associated with the face attributes.
  • the face reconstruction device provided by the embodiment of the present application can execute the face reconstruction method provided by any embodiment of the present application, and has functional modules and effects corresponding to the execution method.
  • FIG. 4 is a schematic structural diagram of a computer device in the fourth embodiment of the present application.
  • Figure 4 shows a block diagram of an exemplary device 12 suitable for implementing embodiments of the present application.
  • the device 12 shown in FIG. 4 is only an example, and should not bring any limitation to the function and scope of use of the embodiments of the present application.
  • the device 12 is represented in the form of a general-purpose computing device.
  • the components of the device 12 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 connecting different system components (including the system memory 28 and the processing unit 16).
  • the bus 18 represents one or more of several types of bus structures, including a memory bus or a memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any bus structure among multiple bus structures.
  • these architectures include, but are not limited to, Industry Standard Architecture (Subversive Alliance, ISA) bus, Micro Channel Architecture (MCA) bus, enhanced ISA bus, Video Electronics Standards Association (Video Electronics Standards) Association, VESA) local bus and Peripheral Component Interconnect (PCI) bus.
  • the device 12 includes a variety of computer system readable media. These media can be any available media that can be accessed by the device 12, including volatile and non-volatile media, removable and non-removable media.
  • the system memory 28 may include a computer system readable medium in the form of a volatile memory, such as a random access memory (RAM) 30 and/or a cache 32.
  • the device 12 may include other removable/non-removable, volatile/non-volatile computer system storage media.
  • the storage system 34 may be used to read and write non-removable, non-volatile magnetic media (not shown in FIG. 4, usually referred to as a "hard drive").
  • a disk drive for reading and writing to a removable non-volatile disk (such as a "floppy disk"), and a removable non-volatile optical disk (such as a portable compact disk read-only memory (Compact Disk)).
  • the system memory 28 may include at least one program product, the program product having a set (for example, at least one) program modules, and these program modules are configured to perform the functions of multiple embodiments of the present application.
  • a program/utility tool 40 having a set of (at least one) program module 42 may be stored in, for example, the system memory 28.
  • Such program module 42 includes but is not limited to an operating system, one or more application programs, other program modules, and programs Data, each of these examples or a combination may include the realization of a network environment.
  • the program module 42 usually executes the functions and/or methods in the embodiments described in this application.
  • the device 12 may also communicate with one or more external devices 14 (such as keyboards, pointing devices, displays 24, etc.), and may also communicate with one or more devices that enable a user to interact with the device 12, and/or communicate with
  • the device 12 can communicate with any device (such as a network card, modem, etc.) that can communicate with one or more other computing devices. This communication can be performed through an input/output (Input/Output, I/O) interface 22.
  • the device 12 may also communicate with one or more networks (for example, a local area network (LAN), a wide area network (WAN), and/or a public network, such as the Internet) through the network adapter 20. As shown in the figure, the network adapter 20 communicates with other modules of the device 12 through the bus 18.
  • LAN local area network
  • WAN wide area network
  • public network such as the Internet
  • microcode can be used in conjunction with the device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, and disk arrays (Redundant Arrays of Independent Disks, RAID) systems, tape drives, and data backup storage systems.
  • device drivers can be used in conjunction with the device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, and disk arrays (Redundant Arrays of Independent Disks, RAID) systems, tape drives, and data backup storage systems.
  • RAID Redundant Arrays of Independent Disks
  • the processing unit 16 executes a variety of functional applications and data processing by running a program stored in the system memory 28, such as realizing the face reconstruction method provided by the embodiment of the present application.
  • a face reconstruction method including:
  • the face reconstruction parameters generate a reconstructed face that matches the face of the anchor
  • the fifth embodiment of the present application also discloses a computer storage medium storing a computer program, and when the computer program is executed by a processor, a face reconstruction method is realized, including:
  • the face reconstruction parameters generate a reconstructed face that matches the face of the anchor
  • the computer storage medium of the embodiment of the present application may adopt any combination of one or more computer-readable media.
  • the computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium.
  • the computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or a combination of any of the above.
  • Computer-readable storage media include: electrical connections with one or more wires, portable computer disks, hard disks, RAM, ROM, Erasable Programmable Read-Only Memory (EPROM), flash memory, optical fiber, CD-ROM, optical storage device, magnetic storage device, or any suitable combination of the above.
  • the computer-readable storage medium can be any tangible medium that contains or stores a program, and the program can be used by or in combination with an instruction execution system, apparatus, or device.
  • the computer-readable signal medium may include a data signal propagated in baseband or as a part of a carrier wave, and computer-readable program code is carried therein. This propagated data signal can take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing.
  • the computer-readable signal medium may also be any computer-readable medium other than the computer-readable storage medium.
  • the computer-readable medium may send, propagate, or transmit the program for use by or in combination with the instruction execution system, apparatus, or device .
  • the program code contained on the computer-readable medium can be transmitted by any suitable medium, including but not limited to wireless, wire, optical cable, radio frequency (RF), etc., or any suitable combination of the foregoing.
  • suitable medium including but not limited to wireless, wire, optical cable, radio frequency (RF), etc., or any suitable combination of the foregoing.
  • the computer program code used to perform the operations of this application can be written in one or more programming languages or a combination thereof.
  • the programming languages include object-oriented programming languages—such as Java, Smalltalk, C++, and also conventional procedural programming languages. Programming language-such as "C" language or similar programming language.
  • the program code can be executed entirely on the user's computer, partly on the user's computer, executed as an independent software package, partly on the user's computer and partly executed on a remote computer, or entirely executed on the remote computer or server.
  • the remote computer may be connected to the user's computer through any kind of network including LAN or WAN, or may be connected to an external computer (for example, using an Internet service provider to connect through the Internet).

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Health & Medical Sciences (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Health & Medical Sciences (AREA)
  • Human Computer Interaction (AREA)
  • Databases & Information Systems (AREA)
  • Signal Processing (AREA)
  • Image Processing (AREA)

Abstract

La présente invention concerne un procédé et un appareil de reconstruction de visage, un dispositif informatique et un support de stockage. Le procédé de reconstruction de visage comprend : l'acquisition de paramètres de reconstruction de visage réglés par un public cible pour le visage d'un présentateur dans un flux vidéo en direct (110) ; selon les paramètres de reconstruction de visage, la génération d'un visage reconstruit qui correspond au visage du présentateur (120) ; et après que le visage du présentateur dans le flux vidéo en direct a été remplacé par le visage reconstruit, la fourniture de ce dernier au public cible à des fins d'affichage (130).
PCT/CN2021/079934 2020-03-11 2021-03-10 Procédé et appareil de reconstruction de visage, dispositif informatique et support de stockage WO2021180114A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010165492.5 2020-03-11
CN202010165492.5A CN111402352B (zh) 2020-03-11 2020-03-11 人脸重构方法、装置、计算机设备及存储介质

Publications (1)

Publication Number Publication Date
WO2021180114A1 true WO2021180114A1 (fr) 2021-09-16

Family

ID=71432279

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/079934 WO2021180114A1 (fr) 2020-03-11 2021-03-10 Procédé et appareil de reconstruction de visage, dispositif informatique et support de stockage

Country Status (2)

Country Link
CN (1) CN111402352B (fr)
WO (1) WO2021180114A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113627404A (zh) * 2021-10-12 2021-11-09 中国科学院自动化研究所 基于因果推断的高泛化人脸替换方法、装置和电子设备

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111402352B (zh) * 2020-03-11 2024-03-05 广州虎牙科技有限公司 人脸重构方法、装置、计算机设备及存储介质

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140043329A1 (en) * 2011-03-21 2014-02-13 Peng Wang Method of augmented makeover with 3d face modeling and landmark alignment
CN108040285A (zh) * 2017-11-15 2018-05-15 上海掌门科技有限公司 视频直播画面调整方法、计算机设备及存储介质
CN108197555A (zh) * 2017-12-28 2018-06-22 杭州相芯科技有限公司 一种基于人脸追踪的实时人脸融合方法
CN109325549A (zh) * 2018-10-25 2019-02-12 电子科技大学 一种人脸图像融合方法
CN110418146A (zh) * 2018-04-27 2019-11-05 武汉斗鱼网络科技有限公司 应用于直播场景的换脸方法、存储介质、电子设备及系统
CN110809171A (zh) * 2019-11-12 2020-02-18 腾讯科技（深圳）有限公司 视频处理方法及相关设备
CN111402352A (zh) * 2020-03-11 2020-07-10 广州虎牙科技有限公司 人脸重构方法、装置、计算机设备及存储介质

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113329252B (zh) * 2018-10-24 2023-01-06 广州虎牙科技有限公司 一种基于直播的人脸处理方法、装置、设备和存储介质

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140043329A1 (en) * 2011-03-21 2014-02-13 Peng Wang Method of augmented makeover with 3d face modeling and landmark alignment
CN108040285A (zh) * 2017-11-15 2018-05-15 上海掌门科技有限公司 视频直播画面调整方法、计算机设备及存储介质
CN108197555A (zh) * 2017-12-28 2018-06-22 杭州相芯科技有限公司 一种基于人脸追踪的实时人脸融合方法
CN110418146A (zh) * 2018-04-27 2019-11-05 武汉斗鱼网络科技有限公司 应用于直播场景的换脸方法、存储介质、电子设备及系统
CN109325549A (zh) * 2018-10-25 2019-02-12 电子科技大学 一种人脸图像融合方法
CN110809171A (zh) * 2019-11-12 2020-02-18 腾讯科技（深圳）有限公司 视频处理方法及相关设备
CN111402352A (zh) * 2020-03-11 2020-07-10 广州虎牙科技有限公司 人脸重构方法、装置、计算机设备及存储介质

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113627404A (zh) * 2021-10-12 2021-11-09 中国科学院自动化研究所 基于因果推断的高泛化人脸替换方法、装置和电子设备

Also Published As

Publication number Publication date
CN111402352B (zh) 2024-03-05
CN111402352A (zh) 2020-07-10

Similar Documents

Publication Publication Date Title
US10699482B2 (en) Real-time immersive mediated reality experiences
Orduna et al. Video multimethod assessment fusion (VMAF) on 360VR contents
Gutiérrez et al. Introducing un salient360! benchmark: A platform for evaluating visual attention models for 360 contents
KR102509398B1 (ko) 비디오 회의에서의 감정 인식
US20210392296A1 (en) Predicting behavior changes of a participant of a 3d video conference
WO2021229415A1 (fr) Procédé et système destinés à des communications virtuelles en 3d
CN106682632B (zh) 用于处理人脸图像的方法和装置
US8842154B2 (en) Systems and methods for providing personal video services
WO2021180114A1 (fr) Procédé et appareil de reconstruction de visage, dispositif informatique et support de stockage
US8908987B1 (en) Providing image candidates based on diverse adjustments to an image
US20230123005A1 (en) Real-time video dimensional transformations of video for presentation in mixed reality-based virtual spaces
US20230146178A1 (en) Attention based audio adjustment in virtual environments
US10642881B2 (en) System architecture for universal emotive autography
US11790535B2 (en) Foreground and background segmentation related to a virtual three-dimensional (3D) video conference
CN111429338B (zh) 用于处理视频的方法、装置、设备和计算机可读存储介质
US11134217B1 (en) System that provides video conferencing with accent modification and multiple video overlaying
DE102018130085A1 (de) Erzeugen eines 2d-videos aus einem 360-video
Covaci et al. Multisensory 360 videos under varying resolution levels enhance presence
CN113900522A (zh) 一种虚拟形象的互动方法、装置
US20230106330A1 (en) Method for creating a variable model of a face of a person
CN115880400A (zh) 卡通数字人的形象生成方法、装置、电子设备及介质
CN111415397B (zh) 一种人脸重构、直播方法、装置、设备及存储介质
CN114419253A (zh) 一种卡通人脸的构建、直播方法及相关装置
Ozcinar et al. Delivery of omnidirectional video using saliency prediction and optimal bitrate allocation
CN110136272B (zh) 用于向远程用户虚拟应用彩妆效果的系统和方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21768688

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21768688

Country of ref document: EP

Kind code of ref document: A1