WO2021227916A1 - 面部形象生成方法、装置、电子设备及可读存储介质 - Google Patents

面部形象生成方法、装置、电子设备及可读存储介质 Download PDF

Info

Publication number
WO2021227916A1
WO2021227916A1 PCT/CN2021/091859 CN2021091859W WO2021227916A1 WO 2021227916 A1 WO2021227916 A1 WO 2021227916A1 CN 2021091859 W CN2021091859 W CN 2021091859W WO 2021227916 A1 WO2021227916 A1 WO 2021227916A1
Authority
WO
WIPO (PCT)
Prior art keywords
facial
target user
target
image
expression
Prior art date
Application number
PCT/CN2021/091859
Other languages
English (en)
French (fr)
Inventor
欧阳灵趋
陈迪凡
Original Assignee
维沃移动通信有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 维沃移动通信有限公司 filed Critical 维沃移动通信有限公司
Priority to KR1020227042575A priority Critical patent/KR20230006009A/ko
Priority to EP21804806.4A priority patent/EP4148677A4/en
Priority to JP2022567137A priority patent/JP2023524119A/ja
Publication of WO2021227916A1 publication Critical patent/WO2021227916A1/zh
Priority to US17/984,212 priority patent/US20230085099A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/00—2D [Two Dimensional] image generation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/012—Head tracking input arrangements
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/60—Analysis of geometric attributes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/174—Facial expression recognition
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
    • G10L25/90—Pitch determination of speech signals
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30196—Human being; Person
    • G06T2207/30201—Face
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/161—Detection; Localisation; Normalisation
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
    • G10L25/48—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use
    • G10L25/51—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination
    • G10L25/63—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination for estimating an emotional state

Definitions

  • the embodiments of the application relate to the field of communications, and in particular to a method, device, electronic device, and readable storage medium for generating a facial image.
  • AR Augmented Reality
  • users need to wear an AR device, which will occlude the user's face, making it impossible to see the user's face, and it is difficult to distinguish each other's identities.
  • the current processing method is to replace the user image through the pre-recorded avatar and the capture of user actions, but the preset avatar is relatively fixed, and the face only has simple speaking actions, unable to make expressions, resulting in the user's AR
  • the interactive experience is poor.
  • the purpose of the embodiments of the present application is to provide a facial image generation method, device, electronic device, and readable storage medium, which can solve the problem that users cannot display facial images in existing AR interactive scenes, resulting in poor AR interactive experience.
  • an embodiment of the present application provides a method for generating a facial image, which is applied to an electronic device, and the method includes:
  • the facial image of the target user is generated.
  • an embodiment of the present application provides a facial image generation device, which is applied to an electronic device, and includes:
  • the first acquiring module is used to acquire the facial contour of the target user wearing the AR device
  • a determining module configured to determine a facial angle of the target user according to the facial contour, the facial angle including the angle between the facial orientation of the target user and the electronic device;
  • the second acquisition module is used to acquire the target facial expression
  • the generating module is used to generate the facial image of the target user according to the target facial expression and the facial angle.
  • an embodiment of the present application provides an electronic device, including a processor, a memory, and a program or instruction stored on the memory and capable of running on the processor, and the program or instruction is executed by the processor. When executed, the steps of the facial image generation method as described in the first aspect are realized.
  • an embodiment of the present application provides a readable storage medium on which a program or instruction is stored, and when the program or instruction is executed by a processor, the facial image generation method as described in the first aspect is implemented A step of.
  • an embodiment of the present application provides a chip, the chip includes a processor and a communication interface, the communication interface is coupled to the processor, and the processor is used to run a program or an instruction to implement the chip as in the first aspect The method described.
  • embodiments of the present application provide a computer software product, the computer software product is stored in a non-volatile storage medium, and the software product is configured to be executed by at least one processor to implement the computer software product as described in the first aspect.
  • the steps of the facial image generation method are described in the first aspect.
  • an embodiment of the present application provides a facial image generating device configured to execute the facial image generating method as described in the first aspect.
  • the facial angle of the target user is determined according to the facial contour of the target user wearing the AR device, and the facial image of the target user is generated based on the facial angle of the target user and the acquired target facial expressions.
  • the user's facial image can be displayed, and according to different target facial expressions, the user can express through different expressions to improve the AR interactive experience.
  • FIG. 1 is a schematic flowchart of a facial image generation method provided by an embodiment of the application
  • Figure 2a is one of the schematic diagrams of application scenarios provided by an embodiment of the application.
  • Figure 2b is the second schematic diagram of the application scenario provided by the embodiment of the application.
  • FIG. 3 is a schematic structural diagram of a facial image generation device provided by an embodiment of the application.
  • FIG. 4 is a schematic structural diagram of an electronic device provided by an embodiment of the application.
  • the AR interactive scene can be an AR-based conference, that is, participants wear AR devices to discuss and interact in the meeting, or the AR interactive scene can also be a remote role-playing game or a lecture to children in a remote place.
  • Casual scenes such as stories.
  • An embodiment of the present application provides a method for generating a facial image, and the method is applied to an electronic device.
  • the above-mentioned electronic device is a device with a camera, based on the image obtained by the camera, combined with AR technology for AR interaction, the electronic device can be an AR device, such as AR glasses, etc., the electronic device can also be a mobile phone, a computer, etc., the embodiment of the present application
  • the specific type of electronic equipment is not limited.
  • the method includes:
  • Step 101 Obtain the facial contour of the target user wearing the AR device
  • the target user refers to an object that performs AR interaction.
  • the target user may be a speaker wearing an AR device.
  • the facial contour of the target user can be obtained through an AR device with a camera, or it can be obtained by a mobile phone, a computer and other devices with a camera, or it can be obtained through a separate setting.
  • a camera captures the facial contours of the target user, and then transmits the captured images to the device for display.
  • acquiring the facial contour of the target user wearing the AR device specifically includes:
  • the closest point of the face refers to the point where the face of the target user is closest to the camera when the face image of the target user is acquired through the camera.
  • the closest point of the face corresponds to the target The position of the tip of the user's nose.
  • the aforementioned preset distance corresponds to a distance of one head width of the target user, and the distance of one head width can be set based on the average head width of a person.
  • the facial contour of the target user can reflect the facial shape and the angle of the face.
  • Step 102 Determine the face angle of the target user according to the facial contour
  • the face angle includes the angle between the face orientation of the target user and the electronic device.
  • the facial angle is used to reflect the facial posture of the target user's face relative to the electronic device. For example, according to the facial contour of the target user, it is determined that the target user's face is 30 degrees upwards relative to the direct counterpart of the electronic device. This can be reflected When the target user's facial posture is 30 degrees upside down. Further, for more complex facial angles, for example, the target user’s face is inclined obliquely, the deflection angle of the target user’s face relative to the electronic device in multiple directions (for example, horizontal and vertical) can be determined based on the current situation. Some spatial angle algorithms calculate the face angle of the target user.
  • determining the facial angle of the target user according to the facial contour specifically includes: matching the facial angle corresponding to the facial contour in the facial contour library; wherein the facial contour library includes pre-recorded target users from multiple facial angles Facial contours.
  • the facial contours of the target user can be recorded from multiple angles in advance, and a facial contour library is generated according to the recorded facial contours.
  • a facial contour library different facial contours correspond to different facial angles. After obtaining the facial contour of the target user, the corresponding facial angle can be directly matched in the facial contour library.
  • Step 103 Obtain the target facial expression
  • the target facial expression refers to the facial expression used to generate the facial image of the target user.
  • the target facial expression can be the expression of the target user, the expression of other users, or it can be obtained from the network. Some cartoon expressions.
  • Step 104 Generate a facial image of the target user according to the target facial expression and facial angle.
  • the selected target facial expressions are generated according to the determined facial angles to generate the facial image of the target user, so that the angle of the generated facial image can be consistent with the current facial angle of the target user, making the facial image closer The face of the target user.
  • the facial angle of the target user is determined according to the facial contour of the target user wearing the AR device, and the facial image of the target user is generated based on the facial angle of the target user and the acquired target facial expressions.
  • the user's facial image can be displayed, and according to different target facial expressions, the user can express through different expressions to improve the AR interactive experience.
  • acquiring the target facial expression described above specifically includes: acquiring the facial expression of the target user recorded by the AR device;
  • the AR device worn by the target user has an image acquisition function, for example: the AR device is AR glasses with a camera
  • the AR device worn by the target user can directly perform facial expressions on the target user. Record and scan in real time, and use the recorded facial expressions as target facial expressions to generate the facial image of the target user.
  • the above-mentioned acquiring target facial expressions specifically includes: acquiring input information of the target user, and according to the context of the input information, matching facial expressions corresponding to the context from the facial expression database; wherein, facial expressions
  • the expression library includes a variety of facial expressions at multiple facial angles.
  • various expressions of the target user from various angles can be recorded in advance, such as expressionless, angry, happy, depressed, Sad, serious, questioning, etc., set a corresponding label for each expression accordingly.
  • the input information of the target user is obtained.
  • the input information can be the text input by the target user or the voice input by the target user.
  • the context recognition is performed based on the input information, and then the facial expression database is displayed in the facial expression database according to the context. Select the corresponding tags and match the corresponding facial expressions. For example, if the target user says "I am very happy today", it will be recognized according to the context. At this time, the context should correspond to the happy tag, and then select the band from the facial expression library. Facial expressions with happy tags.
  • the facial expression library can also store a variety of facial expressions of some other objects at multiple facial angles, such as cats, dogs and other animals, superman, steel and other cartoon characters, which can be used in remote role-playing games. Or in casual scenes such as telling a story to a child in a different place, the facial image of the target user can be replaced with a cartoon image to improve the AR interaction effect.
  • the method further includes at least one of the following:
  • the target user’s mouth state is used to reflect the target user’s mouth actions, such as: opening mouth, closing mouth, pursing mouth, mouth tilted to one side, etc.
  • the adjustment of the mouth state enables the target facial expressions to accurately and synchronously show the subtle expressions of the target user, making the facial expressions richer.
  • the voice of the target user is called to reflect the emotional state of the target user, and can then be used to determine the magnitude of the target expression. For example, it can be divided into 1-10 degrees to correspond to the recorded expression from no expression to happy expression. In the 10 moments, as the pitch of the target user’s voice becomes higher, the emoji used is more backward. In this way, the expression amplitude of the target facial expression is adjusted through the voice tone, so that the target facial expression can accurately show the emotional state of the target user, and the facial expressions are richer.
  • the target user wearing the AR device has multiple people, and facial image generation is required. Then these target users can bind the facial image generation function through preset gesture actions. For example, the AR device will randomly generate a set of gestures, such as stone, cloth, and scissors. Those who need to bind the facial image generation function need to do so. Raise your hands and make gestures in the specified order to confirm. This can make the facial image generation in the AR interaction process more flexible.
  • the target user needs to temporarily handle other affairs.
  • the target user can lock his current facial image, so as to avoid processing other affairs.
  • the change of facial image affects other people. For example, in the scene of AR meeting, one of the participants needs to deal with other affairs temporarily, then he can lock his facial image to prevent facial image from happening when dealing with other affairs. The change made other participants mistakenly express his opinions.
  • the method further includes: receiving a first input of the target user; in response to the first input, replacing the facial image of the target user with a preset image; the preset image includes at least one of the following : Cartoon emoticon image; an image corresponding to the input information of the target user.
  • the image corresponding to the input information can be in the form of an icon.
  • the target user can replace the facial image with a preset image.
  • a preset image For example, referring to Figure 2a, replace the facial image with a cartoon emoticon image, such as emoji, a popular emoticon package on the Internet, etc.
  • a cartoon emoticon image such as emoji, a popular emoticon package on the Internet, etc.
  • a "check" or "fork” icon is used to replace the facial image.
  • the execution subject may be a facial image generation device, or a control module in the facial image generation device for executing the method of loading the facial image generation.
  • the facial image generation apparatus executes the method for generating a loaded facial image as an example to illustrate the facial image generation provided by the embodiment of the present application.
  • an embodiment of the present application provides a facial image generating apparatus 300, which is applied to electronic equipment, and includes:
  • the first acquiring module 301 is configured to acquire the facial contour of the target user wearing the AR device;
  • the determining module 302 is configured to determine a facial angle of the target user according to the facial contour, where the facial angle includes the angle between the facial orientation of the target user and the electronic device;
  • the second acquiring module 303 is used to acquire target facial expressions
  • the generating module 304 is configured to generate the facial image of the target user according to the target facial expression and the facial angle.
  • the first obtaining module 301 includes:
  • the first matching unit is configured to match the facial angle corresponding to the facial contour in the facial contour library
  • the facial contour library includes the facial contours of the target user recorded in advance from multiple facial angles.
  • the second obtaining module 303 includes:
  • the first acquiring unit is configured to acquire the facial expression of the target user recorded by the AR device
  • the second acquiring unit is used to acquire the input information of the target user
  • the second matching unit is configured to match facial expressions corresponding to the context from the facial expression database according to the context of the input information
  • the facial expression library includes multiple facial expressions at multiple facial angles.
  • the device 300 further includes:
  • the third obtaining module is used to obtain the mouth state of the target user
  • the first adjustment module is configured to adjust the mouth state of the target facial expression according to the mouth state of the target user
  • the fourth acquiring module is used to acquire the voice pitch of the target user
  • the first adjustment module is configured to adjust the expression amplitude of the target facial expression according to the voice pitch.
  • the device 300 further includes:
  • a receiving module for receiving the first input of the target user
  • a replacement module configured to replace the facial image of the target user with a preset image in response to the first input
  • the preset image includes at least one of the following:
  • the facial angle of the target user is determined according to the facial contour of the target user wearing the AR device, and the facial image of the target user is generated based on the facial angle of the target user and the acquired target facial expressions.
  • the user's facial image can be displayed, and according to different target facial expressions, the user can express through different expressions to improve the AR interactive experience.
  • the facial image generation device in the embodiment of the present application may be a device, or a component, integrated circuit, or chip in a terminal.
  • the device can be a mobile electronic device or a non-mobile electronic device.
  • the mobile electronic device may be a mobile phone, a tablet computer, a notebook computer, a handheld computer, a vehicle electronic device, a wearable device, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (personal digital assistant).
  • UMPC ultra-mobile personal computer
  • netbook or a personal digital assistant (personal digital assistant).
  • non-mobile electronic devices can be servers, network attached storage (NAS), personal computers (PC), televisions (television, TV), teller machines or self-service machines, etc., this application
  • NAS network attached storage
  • PC personal computers
  • TV televisions
  • teller machines or self-service machines etc.
  • the facial image generation device in the embodiment of the present application may be a device with an operating system.
  • the operating system may be an Android operating system, an ios operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.
  • the facial image generating device provided by the embodiment of the present application can implement the various processes implemented by the facial image generating device in the method embodiments of FIGS. 1 to 2b. In order to avoid repetition, details are not described here
  • an embodiment of the present application further provides an electronic device, including a processor 410, a memory 409, a program or instruction that is stored on the memory 409 and can run on the processor 410, and the program or instruction is executed by the processor 410.
  • an electronic device including a processor 410, a memory 409, a program or instruction that is stored on the memory 409 and can run on the processor 410, and the program or instruction is executed by the processor 410.
  • the electronic devices in the embodiments of the present application include the above-mentioned mobile electronic devices and non-mobile electronic devices.
  • FIG. 4 is a schematic diagram of the hardware structure of an electronic device that implements an embodiment of the present application.
  • the electronic device 400 includes but is not limited to: a radio frequency unit 401, a network module 402, an audio output unit 403, an input unit 404, a sensor 405, a display unit 406, a user input unit 407, an interface unit 408, a memory 409, a processor 410, etc. part.
  • the electronic device 400 may also include a power source (such as a battery) for supplying power to various components.
  • the power source may be logically connected to the processor 410 through a power management system, so that the power management system can manage charging, discharging, and power management. Consumption management and other functions.
  • the structure of the electronic device shown in FIG. 4 does not constitute a limitation on the electronic device.
  • the electronic device may include more or less components than those shown in the figure, or some components may be combined, or different component arrangements, which will not be repeated here. .
  • the input unit 404 may be a camera, which is used to obtain the facial contour of the target user wearing the AR device;
  • the processor 410 is configured to determine a facial angle of the target user according to the facial contour, where the facial angle includes the angle between the facial orientation of the target user and the electronic device;
  • the input unit 404 is also used to obtain target facial expressions
  • the processor 410 is further configured to generate a facial image of the target user according to the target facial expression and the facial angle.
  • the facial angle of the target user is determined according to the facial contour of the target user wearing the AR device, and the facial image of the target user is generated based on the facial angle of the target user and the acquired target facial expressions.
  • the user's facial image can be displayed, and according to different target facial expressions, the user can express through different expressions to improve the AR interactive experience.
  • the processor 410 is further configured to match a facial angle corresponding to the facial contour in a facial contour library;
  • the facial contour library includes the facial contours of the target user recorded in advance from multiple facial angles.
  • the input unit 404 is further configured to obtain the facial expression of the target user recorded by the AR device; or, obtain the input information of the target user, and obtain the input information from the facial expression according to the context of the input information. Match facial expressions corresponding to the context in the library;
  • the facial expression library includes multiple facial expressions at multiple facial angles.
  • the input unit 404 is further configured to obtain the mouth state of the target user, and adjust the mouth state of the target facial expression according to the mouth state of the target user;
  • the input unit 404 is also used to obtain the voice pitch of the target user, and adjust the expression amplitude of the target facial expression according to the voice pitch.
  • the user input unit 407 is configured to receive the first input of the target user
  • the processor 410 is further configured to, in response to the first input, replace the facial image of the target user with a preset image; the preset image includes at least one of the following: a cartoon emoticon image; The image corresponding to the input information.
  • the embodiment of the present application also provides a readable storage medium with a program or instruction stored on the readable storage medium.
  • the program or instruction is executed by a processor, each process of the above-mentioned facial image generation method embodiment is realized, and can achieve The same technical effect, in order to avoid repetition, will not be repeated here.
  • the processor is the processor in the electronic device described in the foregoing embodiment.
  • the readable storage medium includes a computer readable storage medium, such as a computer read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk, or optical disk.
  • the embodiment of the present application further provides a chip, the chip includes a processor and a communication interface, the communication interface is coupled to the processor, and the processor is used to run a program or an instruction to implement the above embodiment of the facial image generation method
  • the chip includes a processor and a communication interface
  • the communication interface is coupled to the processor
  • the processor is used to run a program or an instruction to implement the above embodiment of the facial image generation method
  • chips mentioned in the embodiments of the present application may also be referred to as system-level chips, system-on-chips, system-on-chips, or system-on-chips.
  • the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments.
  • the functional units in the various embodiments of the present disclosure may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit.
  • the technical solution of this application essentially or the part that contributes to the existing technology can be embodied in the form of a software product, and the computer software product is stored in a storage medium (such as ROM/RAM, magnetic disk, The optical disc) includes several instructions to make a terminal (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) execute the methods described in the various embodiments of the present application.
  • a terminal which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.
  • the program can be stored in a computer readable storage medium. When executed, it may include the procedures of the above-mentioned method embodiments.
  • the storage medium may be a magnetic disk, an optical disc, a read-only memory (Read-Only Memory, ROM), or a random access memory (Random Access Memory, RAM), etc.
  • modules, units, and sub-units can be implemented in one or more application specific integrated circuits (ASIC), digital signal processors (Digital Signal Processor, DSP), and digital signal processing equipment (DSP Device, DSPD). ), programmable logic devices (Programmable Logic Device, PLD), Field-Programmable Gate Array (FPGA), general-purpose processors, controllers, microcontrollers, microprocessors, used to implement this disclosure Other electronic units or a combination of the functions described above.
  • ASIC application specific integrated circuits
  • DSP Digital Signal Processor
  • DSP Device digital signal processing equipment
  • DSPD digital signal processing equipment
  • PLD programmable logic devices
  • FPGA Field-Programmable Gate Array
  • the technology described in the embodiments of the present disclosure can be implemented by modules (for example, procedures, functions, etc.) that perform the functions described in the embodiments of the present disclosure.
  • the software codes can be stored in the memory and executed by the processor.
  • the memory can be implemented in the processor or external to the processor.

Abstract

一种面部形象生成方法、装置、电子设备及可读存储介质，属于通信领域。该方法应用于电子设备，包括：获取佩戴AR设备的目标用户的面部轮廓（101）；根据面部轮廓确定目标用户的面部角度，面部角度包括目标用户的面部朝向与电子设备之间的角度（102）；获取目标面部表情（103）；根据目标面部表情和面部角度，生成目标用户的面部形象（104）。

Description

面部形象生成方法、装置、电子设备及可读存储介质
相关申请的交叉引用
本申请主张在2020年5月9日在中国提交的中国专利申请号No.202010386661.8的优先权，其全部内容通过引用包含于此。
技术领域
本申请实施例涉及通信领域，具体涉及一种面部形象生成方法、装置、电子设备及可读存储介质。
背景技术
在利用现有增强现实(Augmented Reality，AR)技术进行互动的场景中，用户需要佩戴AR设备，该AR设备会对用户的面部造成遮挡，导致无法看到用户的面部，难以分辨彼此的身份。目前的处理方式为，通过预先录制的虚拟形象以及对用户动作的捕捉进行用户形象替换，但预设的虚拟形象相对固定，且面部只具备简单的说话动作，无法做出表情，导致用户的AR互动体验较差。
发明内容
本申请实施例的目的是提供一种面部形象生成方法、装置、电子设备及可读存储介质，能够解决现有AR互动场景中，用户无法展现面部形象，导致AR互动体验较差的问题。
为了解决上述技术问题，本申请是这样实现的：
第一方面，本申请实施例提供一种面部形象生成方法，应用于电子设备，所述方法包括：
获取佩戴AR设备的目标用户的面部轮廓；
根据所述面部轮廓确定所述目标用户的面部角度，所述面部角度包括所述目标用户的面部朝向与所述电子设备之间的角度；
获取目标面部表情；
根据所述目标面部表情和所述面部角度，生成所述目标用户的面部形象。
第二方面，本申请实施例提供一种面部形象生成装置，应用于电子设备，包括：
第一获取模块，用于获取佩戴AR设备的目标用户的面部轮廓；
确定模块，用于根据所述面部轮廓确定所述目标用户的面部角度，所述面部角度包括所述目标用户的面部朝向与所述电子设备之间的角度；
第二获取模块，用于获取目标面部表情；
生成模块，用于根据所述目标面部表情和所述面部角度，生成所述目标用户的面部形象。
第三方面，本申请实施例提供一种电子设备，包括处理器、存储器及存储在所述存储器上并可在所述处理器上运行的程序或指令，所述程序或指令被所述处理器执行时实现如第一方面所述的面部形象生成方法的步骤。
第四方面，本申请实施例提供一种可读存储介质，所述可读存储介质上存储程序或指令，所述程序或指令被处理器执行时实现如第一方面所述的面部形象生成方法的步骤。
第五方面，本申请实施例提供了一种芯片，所述芯片包括处理器和通信接口，所述通信接口和所述处理器耦合，所述处理器用于运行程序或指令，实现如第一方面所述的方法。
第六方面，本申请实施例提供一种计算机软件产品，所述计算机软件产品被存储在非易失的存储介质中，所述软件产品被配置成被至少一个处理器执行以实现如第一方面所述的面部形象生成方法的步骤。
第七方面，本申请实施例提供一种面部形象生成装置，所述面部形象生成装置被配置成用于执行如第一方面所述的面部形象生成方法。
本发明实施例中，根据佩戴AR设备的目标用户的面部轮廓，确定该目标用户的面部角度，基于目标用户的面部角度和获取的目标面部表情生成该目标用户的面部形象。这样在进行AR互动时，能够展现用户的面部形象，且根据不同的目标面部表情，使用户能够通过不同的表情进行表达，提高AR互动体验。
附图说明
图1为本申请实施例提供的面部形象生成方法的流程示意图；
图2a为本申请实施例提供的应用场景示意图之一；
图2b为本申请实施例提供的应用场景示意图之二；
图3为本申请实施例提供的面部形象生成装置的结构示意图；
图4为本申请实施例提供的电子设备的结构示意图。
具体实施方式
下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。
本申请的说明书和权利要求书中的术语“第一”、“第二”等是用于区别类似的对象，而不用于描述特定的顺序或先后次序。应该理解这样使用的数据在适当情况下可以互换，以便本申请的实施例能够以除了在这里图示或描述的那些以外的顺序实施。此外，说明书以及权利要求中“和/或”表示所连接对象的至少其中之一，字符“/”，一般表示前后关联对象是一种“或”的关系。
下面结合附图，通过具体的实施例及其应用场景对本申请实施例提供的面部形象生成方法进行详细地说明。
本申请实施例中，AR互动场景可以是基于AR的会议，即参会者通过佩戴AR设备进行会议讨论和互动，或者AR互动场景也可以是远程的角色扮演游戏中或在给异地的孩子讲故事等休闲场景。
本申请实施例中提供一种面部形象生成方法，该方法应用于电子设备。
上述电子设备为具有摄像头的设备，基于摄像头获取的图像，结合AR技术进行AR互动，该电子设备可以是AR设备，例如AR眼镜等，该电子设备也可以是手机、电脑等，本申请实施例对电子设备的具体类型不做限定。
参见图1，该方法包括：
步骤101：获取佩戴AR设备的目标用户的面部轮廓；
在本申请实施例中，目标用户指的是进行AR互动的对象，例如在AR会议中，目标用户可以是佩戴AR设备的发言人。具体地，获取目标用户的面部轮廓的方式可以包括多种，例如：可以通过带有摄像头的AR设备进行获取，或者可以通过带有摄像头的手机、电脑等设备进行获取，又或者可以通过单独设置一个摄像头对目标用户的面部轮廓进行录取，然后将录取的图像传输到用于显示的装置上。
在一些实施方式中，获取佩戴AR设备的目标用户的面部轮廓，具体包括：
通过摄像头获取目标用户的面部图像，在摄像头与目标用户的面部最近点至该最近点加预设距离的景深范围内，进行多次不同景深的轮廓成像，然后将得到的多个轮廓图像叠加得到该目标用户的面部轮廓。
在本申请实施例中，面部最近点指的是在通过摄像头获取目标用户的面部图像时，目标用户的面部距离摄像头最近的一点，例如：目标用户的面部针对摄像头时，面部最近点对应于目标用户的鼻尖位置。上述预设距离对应于目标用户一个头宽的距离，该一个头宽的距离可以基于人的平均头宽进行设置。
在从面部最近点至面部最近点加一个头宽的景深范围内，分别获取多个不同景深下的轮廓图像，然后基于不同景深下的轮廓图像进行叠加合成，这样能够得到一个相对精确的目标用户的面部轮廓，该面部轮廓能够体现出目标用户的面部形状，以及面部朝向角度。
步骤102：根据面部轮廓确定目标用户的面部角度；
在本申请实施例中，面部角度包括目标用户的面部朝向与电子设备之间的角度。面部角度用于反映目标用户的面部相对于电子设备的面部姿态，例如：根据目标用户的面部轮廓，确定出目标用户的面部朝向相对电子设备的正对方向上扬了30度，则可以反映出此时目标用户的面部姿态为仰面30度。进一步地，针对更复杂的面部角度，例如目标用户的面部呈斜向倾斜，可以通过目标用户的面部朝向相对电子设备在多个方向(例如水平和竖直方向)上的偏转角度，然后基于现有的空间角度算法计算出目标用户的面部角度。
在一些实施方式中，根据面部轮廓确定目标用户的面部角度，具体包括： 在面部轮廓库中匹配与面部轮廓对应的面部角度；其中，面部轮廓库中包括预先从多个面部角度录制的目标用户的面部轮廓。
在本申请实施例中，可以预先从多个角度录制目标用户的面部轮廓，并根据录制的面部轮廓生成一个面部轮廓库，在该面部轮廓库中不同的面部轮廓分别对应不同的面部角度，这样在获取了目标用户的面部轮廓之后，可以直接在面部轮廓库中匹配出对应的面部角度。
步骤103：获取目标面部表情；
在本申请实施例中，目标面部表情指的是用于生成目标用户的面部形象的面部表情，该目标面部表情可以是目标用户的表情，可以是其他用户的表情，还可以是从网络中获取一些卡通形象的表情。
步骤104：根据目标面部表情和面部角度，生成目标用户的面部形象。
在本申请实施例会中，将选取好目标面部表情按照确定出的面部角度生成目标用户的面部形象，这样生成的面部形象的角度与目标用户的当前的面部角度能够保持一致，使面部形象更贴近目标用户的面部。
本发明实施例中，根据佩戴AR设备的目标用户的面部轮廓，确定该目标用户的面部角度，基于目标用户的面部角度和获取的目标面部表情生成该目标用户的面部形象。这样在进行AR互动时，能够展现用户的面部形象，且根据不同的目标面部表情，使用户能够通过不同的表情进行表达，提高AR互动体验。
可选地，在一些实施方式中，上述获取目标面部表情，具体包括：获取由AR设备录制的目标用户的面部表情；
在本申请实施例中，如果目标用户佩戴的AR设备本身具有图像获取功能，例如：该AR设备为带有摄像头的AR眼镜，则可以直接由目标用户佩戴的AR设备对目标用户的面部表情进行实时记录扫描，并将录制到的面部表情作为目标面部表情，用于生成目标用户的面部形象。
可选地，在一些实施方式中，上述获取目标面部表情，具体包括：获取目标用户的输入信息，根据输入信息的语境，从面部表情库中匹配与语境对应的面部表情；其中，面部表情库中包括在多个面部角度下的多种面部表情。
在本申请实施例中，如果目标用户佩戴的AR设备本身并不具有图像获 取功能话，则可以预先录制目标用户在各种角度下的各种表情，例如：无表情、生气、开心、郁闷、伤心、认真、质疑等，相应地针对每种表情设置一个对应的标签。
在进行AR互动时，获取目标用户的输入信息，该输入信息可以是目标用户输入的文字、也可以是目标用户输入的语音，对根据输入信息进行语境识别，然后按照语境在面部表情库选取对应的标签并匹配对应的面部表情，例如：目标用户说了一句“今天我很开心”，则根据语境识别，此时的语境应当对应开心的标签，则从面部表情库中选取带有开心标签的面部表情。
可以理解的是，面部表情库中也可以存储一些其他对象在多个面部角度下的多种面部表情，例如：猫、狗等动物、超人、钢铁等卡通角色，这样在远程的角色扮演游戏中或在给异地的孩子讲故事等休闲场景下，可以将目标用户的面部形象替换为卡通形象，以提高AR互动效果。
进一步地，在根据目标面部表情和面部角度，生成目标用户的面部形象之前，本方法还包括以下至少一项：
(1)获取目标用户的嘴部状态，并根据目标用户的嘴部状态调整目标面部表情的嘴部状态；
在本申请实施例中，目标用户的嘴部状态用于反映目标用户的嘴部动作，例如：张嘴、闭嘴、抿嘴、嘴向一边歪等，通过目标用户的嘴部状态对目标面部表情的嘴部状态调整，使目标面部表情能够准确同步展现出目标用户的细微表情动作，使面部表情更丰富。
(2)获取目标用户的语音音调，并根据语音音调调整目标面部表情的表情幅度；
在本申请实施例中，目标用户的语音音调用于反映目标用户的情绪状态，进而可以用于确定目标表情的幅度，例如：可以分1-10个程度分别对应录制表情从无表情到开心表情的10个瞬间，随着目标用户的语音音调越高，则使用的表情越靠后。这样，通过语音音调调整目标面部表情的表情幅度，使目标面部表情能够准确展现出目标用户的情绪状态，使面部表情更丰富。
可选地，在一些实施方式中，佩戴AR设备的目标用户有多个人，且需要进行面部形象生成。则这些目标用户可以通过预设的手势动作来绑定面部 形象生成功能，例如：AR设备会随机生成一组手势，如石头、布、剪子，需要绑定面部形象生成功能的人此时需要举起手并按指定顺序摆出手势进行确认。这样可以使AR互动过程中的面部形象生成更加灵活。
可选地，在一些实施方式中，目标用户需要临时处理其他事务，为避免对进行AR会议的其他用户造成影响，目标用户可以将自己当前的面部形象进行锁定，从而避免在处理其他事务时，面部形象发生变化对其他人造成影响，例如：在AR会议的场景下，其中一个参会人需要临时处理其他事务，则他可以将自己的面部形象锁定，防止在处理其他事务时，面部形象发生变化，让其他参会人误以外他在发表意见。
可选地，在一些实施方式中，本方法还包括：接收目标用户的第一输入；响应于第一输入，将目标用户的面部形象替换为预设图像；该预设图像包括以下至少一项：卡通表情图像；与目标用户的输入信息对应的图像，该与输入信息对应的图像可以采用图标的形式。
在本申请实施例中，目标用户可以将面部形象替换为预设图像，例如：参见图2a，将面部形象替换为卡通表情图像，如表情符号(emoji)、网络流行的表情包等，以此来达到夸张的表情效果，提高AR会议的趣味性。又例如：参见图2b，在进行投票时直接将结果替换为面部形象，即采用“对勾”或“叉子”的图标替换面部形象。
需要说明的是，本申请实施例提供的面部形象生成方法，执行主体可以为面部形象生成装置，或者该面部形象生成装置中的用于执行加载面部形象生成方法的控制模块。本申请实施例中以面部形象生成装置执行加载面部形象生成方法为例，说明本申请实施例提供的面部形象生成。
参见图3，本申请实施例提供一种面部形象生成装置300，应用于电子设备，包括：
第一获取模块301，用于获取佩戴AR设备的目标用户的面部轮廓；
确定模块302，用于根据所述面部轮廓确定所述目标用户的面部角度，所述面部角度包括所述目标用户的面部朝向与所述电子设备之间的角度；
第二获取模块303，用于获取目标面部表情；
生成模块304，用于根据所述目标面部表情和所述面部角度，生成所述 目标用户的面部形象。
可选地，所述第一获取模块301，包括：
第一匹配单元，用于在面部轮廓库中匹配与所述面部轮廓对应的面部角度；
其中，所述面部轮廓库中包括预先从多个面部角度录制的所述目标用户的面部轮廓。
可选地，所述第二获取模块303，包括：
第一获取单元，用于获取由所述AR设备录制的所述目标用户的面部表情；
第二获取单元，用于获取所述目标用户的输入信息；
第二匹配单元，用于根据所述输入信息的语境，从面部表情库中匹配与所述语境对应的面部表情；
其中，所述面部表情库中包括在多个面部角度下的多种面部表情。
可选地，所述装置300还包括：
第三获取模块，用于获取所述目标用户的嘴部状态；
第一调整模块，用于根据所述目标用户的嘴部状态调整所述目标面部表情的嘴部状态；
第四获取模块，用于获取所述目标用户的语音音调；
第一调整模块，用于根据所述语音音调调整所述目标面部表情的表情幅度。
可选地，所述装置300还包括：
接收模块，用于接收所述目标用户的第一输入；
替换模块，用于响应于所述第一输入，将所述目标用户的面部形象替换为预设图像；
所述预设图像包括以下至少一项：
卡通表情图像；
与所述目标用户的输入信息对应的图像。
本发明实施例中，根据佩戴AR设备的目标用户的面部轮廓，确定该目标用户的面部角度，基于目标用户的面部角度和获取的目标面部表情生成该 目标用户的面部形象。这样在进行AR互动时，能够展现用户的面部形象，且根据不同的目标面部表情，使用户能够通过不同的表情进行表达，提高AR互动体验。
本申请实施例中的面部形象生成装置可以是装置，也可以是终端中的部件、集成电路、或芯片。该装置可以是移动电子设备，也可以为非移动电子设备。示例性的，移动电子设备可以为手机、平板电脑、笔记本电脑、掌上电脑、车载电子设备、可穿戴设备、超级移动个人计算机(ultra-mobile personal computer，UMPC)、上网本或者个人数字助理(personal digital assistant，PDA)等，非移动电子设备可以为服务器、网络附属存储器(Network Attached Storage，NAS)、个人计算机(personal computer，PC)、电视机(television，TV)、柜员机或者自助机等，本申请实施例不作具体限定。
本申请实施例中的面部形象生成装置可以为具有操作系统的装置。该操作系统可以为安卓(Android)操作系统，可以为ios操作系统，还可以为其他可能的操作系统，本申请实施例不作具体限定。
本申请实施例提供的面部形象生成装置能够实现图1至图2b的方法实施例中面部形象生成装置实现的各个过程，为避免重复，这里不再赘述
可选的，本申请实施例还提供一种电子设备，包括处理器410，存储器409，存储在存储器409上并可在所述处理器410上运行的程序或指令，该程序或指令被处理器410执行时实现上述面部形象生成实施例的各个过程，且能达到相同的技术效果，为避免重复，这里不再赘述。
需要注意的是，本申请实施例中的电子设备包括上述所述的移动电子设备和非移动电子设备。
图4为实现本申请实施例的一种电子设备的硬件结构示意图。
该电子设备400包括但不限于：射频单元401、网络模块402、音频输出单元403、输入单元404、传感器405、显示单元406、用户输入单元407、接口单元408、存储器409、以及处理器410等部件。
本领域技术人员可以理解，电子设备400还可以包括给各个部件供电的电源(比如电池)，电源可以通过电源管理系统与处理器410逻辑相连，从而通过电源管理系统实现管理充电、放电、以及功耗管理等功能。图4中示出 的电子设备结构并不构成对电子设备的限定，电子设备可以包括比图示更多或更少的部件，或者组合某些部件，或者不同的部件布置，在此不再赘述。
其中，输入单元404，可以为摄像头，用于获取佩戴AR设备的目标用户的面部轮廓；
处理器410，用于根据所述面部轮廓确定所述目标用户的面部角度，所述面部角度包括所述目标用户的面部朝向与所述电子设备之间的角度；
输入单元404，还用于获取目标面部表情；
处理器410，还用于根据所述目标面部表情和所述面部角度，生成所述目标用户的面部形象。
本发明实施例中，根据佩戴AR设备的目标用户的面部轮廓，确定该目标用户的面部角度，基于目标用户的面部角度和获取的目标面部表情生成该目标用户的面部形象。这样在进行AR互动时，能够展现用户的面部形象，且根据不同的目标面部表情，使用户能够通过不同的表情进行表达，提高AR互动体验。
可选的，处理器410，还用于在面部轮廓库中匹配与所述面部轮廓对应的面部角度；
其中，所述面部轮廓库中包括预先从多个面部角度录制的所述目标用户的面部轮廓。
可选的，输入单元404，还用于获取由所述AR设备录制的所述目标用户的面部表情；或者，获取所述目标用户的输入信息，根据所述输入信息的语境，从面部表情库中匹配与所述语境对应的面部表情；
其中，所述面部表情库中包括在多个面部角度下的多种面部表情。
可选的，输入单元404，还用于获取所述目标用户的嘴部状态，并根据所述目标用户的嘴部状态调整所述目标面部表情的嘴部状态；
输入单元404，还用于获取所述目标用户的语音音调，并根据所述语音音调调整所述目标面部表情的表情幅度。
可选的，用户输入单元407，用于接收所述目标用户的第一输入；
处理器410，还用于响应于所述第一输入，将所述目标用户的面部形象替换为预设图像；所述预设图像包括以下至少一项：卡通表情图像；与所述 目标用户的输入信息对应的图像。
本申请实施例还提供一种可读存储介质，所述可读存储介质上存储有程序或指令，该程序或指令被处理器执行时实现上述面部形象生成方法实施例的各个过程，且能达到相同的技术效果，为避免重复，这里不再赘述。
其中，所述处理器为上述实施例中所述的电子设备中的处理器。所述可读存储介质，包括计算机可读存储介质，如计算机只读存储器(Read-Only Memory，ROM)、随机存取存储器(Random Access Memory，RAM)、磁碟或者光盘等。
本申请实施例另提供了一种芯片，所述芯片包括处理器和通信接口，所述通信接口和所述处理器耦合，所述处理器用于运行程序或指令，实现上述面部形象生成方法实施例的各个过程，且能达到相同的技术效果，为避免重复，这里不再赘述。
应理解，本申请实施例提到的芯片还可以称为系统级芯片、系统芯片、芯片系统或片上系统芯片等。
需要说明的是，在本文中，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者装置不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者装置所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括该要素的过程、方法、物品或者装置中还存在另外的相同要素。此外，需要指出的是，本申请实施方式中的方法和装置的范围不限按示出或讨论的顺序来执行功能，还可包括根据所涉及的功能按基本同时的方式或按相反的顺序来执行功能，例如，可以按不同于所描述的次序来执行所描述的方法，并且还可以添加、省去、或组合各种步骤。另外，参照某些示例所描述的特征可在其他示例中被组合。
本领域普通技术人员可以意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，能够以电子硬件、或者计算机软件和电子硬件的结合来实现。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方 法来实现所描述的功能，但是这种实现不应认为超出本公开的范围。
所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的系统、装置和单元的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外，在本公开各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。
通过以上的实施方式的描述，本领域的技术人员可以清楚地了解到上述实施例方法可借助软件加必需的通用硬件平台的方式来实现，当然也可以通过硬件，但很多情况下前者是更佳的实施方式。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质(如ROM/RAM、磁碟、光盘)中，包括若干指令用以使得一台终端(可以是手机，计算机，服务器，空调器，或者网络设备等)执行本申请各个实施例所述的方法。
本领域普通技术人员可以理解实现上述实施例方法中的全部或部分流程，是可以通过计算机程序来控制相关的硬件来完成，所述的程序可存储于计算机可读取存储介质中，该程序在执行时，可包括如上述各方法的实施例的流程。其中，所述的存储介质可为磁碟、光盘、只读存储器(Read-Only Memory，ROM)或随机存取存储器(Random Access Memory，RAM)等。
可以理解的是，本公开实施例描述的这些实施例可以用硬件、软件、固件、中间件、微码或其组合来实现。对于硬件实现，模块、单元、子单元可以实现在一个或多个专用集成电路(Application Specific Integrated Circuits，ASIC)、数字信号处理器(Digital Signal Processor，DSP)、数字信号处理设备(DSP Device，DSPD)、可编程逻辑设备(Programmable Logic Device，PLD)、现场可编程门阵列(Field-Programmable Gate Array，FPGA)、通用处理器、控 制器、微控制器、微处理器、用于执行本公开所述功能的其它电子单元或其组合中。
对于软件实现，可通过执行本公开实施例所述功能的模块(例如过程、函数等)来实现本公开实施例所述的技术。软件代码可存储在存储器中并通过处理器执行。存储器可以在处理器中或在处理器外部实现。
上面结合附图对本申请的实施例进行了描述，但是本申请并不局限于上述的具体实施方式，上述的具体实施方式仅仅是示意性的，而不是限制性的，本领域的普通技术人员在本申请的启示下，在不脱离本申请宗旨和权利要求所保护的范围情况下，还可做出很多形式，均属于本申请的保护之内。

Claims (15)

  1. 一种面部形象生成方法，应用于电子设备，包括：
    获取佩戴增强现实AR设备的目标用户的面部轮廓；
    根据所述面部轮廓确定所述目标用户的面部角度，所述面部角度包括所述目标用户的面部朝向与所述电子设备之间的角度；
    获取目标面部表情；
    根据所述目标面部表情和所述面部角度，生成所述目标用户的面部形象。
  2. 根据权利要求1所述的方法，其中，所述根据所述面部轮廓确定所述目标用户的面部角度，包括：
    在面部轮廓库中匹配与所述面部轮廓对应的面部角度；
    其中，所述面部轮廓库中包括预先从多个面部角度录制的所述目标用户的面部轮廓。
  3. 根据权利要求1所述的方法，其中，所述获取目标面部表情，包括：
    获取由所述AR设备录制的所述目标用户的面部表情；
    或者，
    获取所述目标用户的输入信息，根据所述输入信息的语境，从面部表情库中匹配与所述语境对应的面部表情；
    其中，所述面部表情库中包括在多个面部角度下的多种面部表情。
  4. 根据权利要求1所述的方法，其中，在根据所述目标面部表情和所述面部角度，生成所述目标用户的面部形象之前，所述方法还包括以下至少一项：
    获取所述目标用户的嘴部状态，并根据所述目标用户的嘴部状态调整所述目标面部表情的嘴部状态；
    获取所述目标用户的语音音调，并根据所述语音音调调整所述目标面部表情的表情幅度。
  5. 根据权利要求1所述的方法，还包括：
    接收所述目标用户的第一输入；
    响应于所述第一输入，将所述目标用户的面部形象替换为预设图像；
    所述预设图像包括以下至少一项：
    卡通表情图像；
    与所述目标用户输入信息对应的图像。
  6. 一种面部形象生成装置，应用于电子设备，包括：
    第一获取模块，用于获取佩戴增强现实AR设备的目标用户的面部轮廓；
    确定模块，用于根据所述面部轮廓确定所述目标用户的面部角度，所述面部角度包括所述目标用户的面部朝向与所述电子设备之间的角度；
    第二获取模块，用于获取目标面部表情；
    生成模块，用于根据所述目标面部表情和所述面部角度，生成所述目标用户的面部形象。
  7. 根据权利要求6所述的装置，其中，所述第一获取模块，包括：
    第一匹配单元，用于在面部轮廓库中匹配与所述面部轮廓对应的面部角度；
    其中，所述面部轮廓库中包括预先从多个面部角度录制的所述目标用户的面部轮廓。
  8. 根据权利要求6所述的装置，其中，所述第二获取模块，包括：
    第一获取单元，用于获取由所述AR设备录制的所述目标用户的面部表情；
    第二获取单元，用于获取所述目标用户的输入信息；
    第二匹配单元，用于根据所述输入信息的语境，从面部表情库中匹配与所述语境对应的面部表情；
    其中，所述面部表情库中包括在多个面部角度下的多种面部表情。
  9. 根据权利要求6所述的装置，还包括：
    第三获取模块，用于获取所述目标用户的嘴部状态；
    第一调整模块，用于根据所述目标用户的嘴部状态调整所述目标面部表情的嘴部状态；
    第四获取模块，用于获取所述目标用户的语音音调；
    第一调整模块，用于根据所述语音音调调整所述目标面部表情的表情幅度。
  10. 根据权利要求6所述的装置，还包括：
    接收模块，用于接收所述目标用户的第一输入；
    替换模块，用于响应于所述第一输入，将所述目标用户的面部形象替换为预设图像；
    所述预设图像包括以下至少一项：
    卡通表情图像；
    与所述目标用户的输入信息对应的图像。
  11. 一种电子设备，包括处理器、存储器及存储在所述存储器上并可在所述处理器上运行的程序或指令，所述程序或指令被所述处理器执行时实现如权利要求1至5中任一项所述的面部形象生成方法的步骤。
  12. 一种可读存储介质，所述可读存储介质上存储程序或指令，所述程序或指令被处理器执行时实现如权利要求1至5中任一项所述的面部形象生成方法的步骤。
  13. 一种芯片，所述芯片包括处理器和通信接口，所述通信接口和所述处理器耦合，所述处理器用于运行程序或指令，实现如权利要求1至5中任一项所述的面部形象生成方法。
  14. 一种计算机软件产品，所述计算机软件产品被存储在非易失的存储介质中，所述软件产品被配置成被至少一个处理器执行以实现如权利要求1至5中任一项所述的面部形象生成方法的步骤。
  15. 一种面部形象生成装置，所述面部形象生成装置被配置成用于执行如权利要求1至5中任一项所述的面部形象生成方法。
PCT/CN2021/091859 2020-05-09 2021-05-06 面部形象生成方法、装置、电子设备及可读存储介质 WO2021227916A1 (zh)

Priority Applications (4)

Application Number Priority Date Filing Date Title
KR1020227042575A KR20230006009A (ko) 2020-05-09 2021-05-06 안면 영상 생성 방법, 장치, 전자기기 및 판독가능 저장 매체
EP21804806.4A EP4148677A4 (en) 2020-05-09 2021-05-06 METHOD AND DEVICE FOR GENERATING FACIAL IMAGE, ELECTRONIC DEVICE AND READABLE STORAGE MEDIUM
JP2022567137A JP2023524119A (ja) 2020-05-09 2021-05-06 顔イメージ生成方法、装置、電子機器及び可読記憶媒体
US17/984,212 US20230085099A1 (en) 2020-05-09 2022-11-09 Facial image generation method and apparatus, electronic device, and readable storage medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010386661.8A CN111583355B (zh) 2020-05-09 2020-05-09 面部形象生成方法、装置、电子设备及可读存储介质
CN202010386661.8 2020-05-09

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/984,212 Continuation US20230085099A1 (en) 2020-05-09 2022-11-09 Facial image generation method and apparatus, electronic device, and readable storage medium

Publications (1)

Publication Number Publication Date
WO2021227916A1 true WO2021227916A1 (zh) 2021-11-18

Family

ID=72124806

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/091859 WO2021227916A1 (zh) 2020-05-09 2021-05-06 面部形象生成方法、装置、电子设备及可读存储介质

Country Status (6)

Country Link
US (1) US20230085099A1 (zh)
EP (1) EP4148677A4 (zh)
JP (1) JP2023524119A (zh)
KR (1) KR20230006009A (zh)
CN (1) CN111583355B (zh)
WO (1) WO2021227916A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115359166A (zh) * 2022-10-20 2022-11-18 北京百度网讯科技有限公司 一种图像生成方法、装置、电子设备和介质

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111583355B (zh) * 2020-05-09 2024-01-23 维沃移动通信有限公司 面部形象生成方法、装置、电子设备及可读存储介质
US20230103125A1 (en) * 2021-09-30 2023-03-30 Snap Inc. 3d upper garment tracking
CN114779948B (zh) * 2022-06-20 2022-10-11 广东咏声动漫股份有限公司 基于面部识别的动画人物即时交互控制方法、装置及设备
CN115946150B (zh) * 2022-12-13 2023-11-14 深圳无芯科技有限公司 具有面部表情的头部结构及具有其的机器人

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107491165A (zh) * 2016-06-12 2017-12-19 张翔宇 一种vr眼镜面部3d图像、平面图像捕获与手势捕获系统
US20180052841A1 (en) * 2016-08-16 2018-02-22 Shanghai Zhangmen Science And Technology Co., Ltd. Method and Device thereof for User Interaction Based on Virtual Objects and Non-volatile Storage Medium
CN108614638A (zh) * 2018-04-23 2018-10-02 太平洋未来科技（深圳）有限公司 Ar成像方法和装置
CN109101953A (zh) * 2018-09-07 2018-12-28 大连东锐软件有限公司 基于人类面部表情的分区要素化的表情动作生成方法
US20200090392A1 (en) * 2018-09-19 2020-03-19 XRSpace CO., LTD. Method of Facial Expression Generation with Data Fusion
CN111583355A (zh) * 2020-05-09 2020-08-25 维沃移动通信有限公司 面部形象生成方法、装置、电子设备及可读存储介质

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8488023B2 (en) * 2009-05-20 2013-07-16 DigitalOptics Corporation Europe Limited Identifying facial expressions in acquired digital images
TW201039251A (en) * 2009-04-30 2010-11-01 Novatek Microelectronics Corp Facial expression recognition apparatus and facial expression recognition method thereof
KR102507567B1 (ko) * 2015-06-09 2023-03-09 삼성전자주식회사 이미지를 처리하는 전자 장치 및 그 제어 방법
KR101810190B1 (ko) * 2016-07-14 2017-12-18 김용상 얼굴 인식을 이용한 사용자 인증 방법 및 그 장치
CN108241434B (zh) * 2018-01-03 2020-01-14 Oppo广东移动通信有限公司 基于景深信息的人机交互方法、装置、介质及移动终端
US10706577B2 (en) * 2018-03-06 2020-07-07 Fotonation Limited Facial features tracker with advanced training for natural rendering of human faces in real-time
EP4262193A3 (en) * 2018-05-07 2023-11-29 Google LLC Puppeteering remote avatar by facial expressions
CN108875633B (zh) * 2018-06-19 2022-02-08 北京旷视科技有限公司 表情检测与表情驱动方法、装置和系统及存储介质
CN109166164B (zh) * 2018-07-25 2023-04-07 维沃移动通信有限公司 一种表情图片的生成方法及终端

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107491165A (zh) * 2016-06-12 2017-12-19 张翔宇 一种vr眼镜面部3d图像、平面图像捕获与手势捕获系统
US20180052841A1 (en) * 2016-08-16 2018-02-22 Shanghai Zhangmen Science And Technology Co., Ltd. Method and Device thereof for User Interaction Based on Virtual Objects and Non-volatile Storage Medium
CN108614638A (zh) * 2018-04-23 2018-10-02 太平洋未来科技（深圳）有限公司 Ar成像方法和装置
CN109101953A (zh) * 2018-09-07 2018-12-28 大连东锐软件有限公司 基于人类面部表情的分区要素化的表情动作生成方法
US20200090392A1 (en) * 2018-09-19 2020-03-19 XRSpace CO., LTD. Method of Facial Expression Generation with Data Fusion
CN111583355A (zh) * 2020-05-09 2020-08-25 维沃移动通信有限公司 面部形象生成方法、装置、电子设备及可读存储介质

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4148677A4 *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115359166A (zh) * 2022-10-20 2022-11-18 北京百度网讯科技有限公司 一种图像生成方法、装置、电子设备和介质

Also Published As

Publication number Publication date
KR20230006009A (ko) 2023-01-10
JP2023524119A (ja) 2023-06-08
CN111583355A (zh) 2020-08-25
EP4148677A4 (en) 2023-11-15
US20230085099A1 (en) 2023-03-16
EP4148677A1 (en) 2023-03-15
CN111583355B (zh) 2024-01-23

Similar Documents

Publication Publication Date Title
WO2021227916A1 (zh) 面部形象生成方法、装置、电子设备及可读存储介质
TWI751161B (zh) 終端設備、智慧型手機、基於臉部識別的認證方法和系統
JP2019145108A (ja) 顔に対応する３次元アバターを用いて顔の動きが反映された３ｄアバターを含むイメージを生成する電子装置
KR20230156408A (ko) 전자 미러링 디바이스를 동작시키는 핸즈-프리 모드를 활성화함
US8044989B2 (en) Mute function for video applications
JP7268071B2 (ja) バーチャルアバターの生成方法及び生成装置
CN110418095B (zh) 虚拟场景的处理方法、装置、电子设备及存储介质
WO2014094199A1 (en) Facial movement based avatar animation
CN110555507B (zh) 虚拟机器人的交互方法、装置、电子设备及存储介质
EP3550812B1 (en) Electronic device and method for delivering message by same
CN110794964A (zh) 虚拟机器人的交互方法、装置、电子设备及存储介质
US20210192192A1 (en) Method and apparatus for recognizing facial expression
JP2020064426A (ja) コミュニケーションシステム及びプログラム
US20230097257A1 (en) Electronic communication interface with haptic feedback response
US20220206581A1 (en) Communication interface with haptic feedback response
US11562548B2 (en) True size eyewear in real time
CN112449098B (zh) 一种拍摄方法、装置、终端及存储介质
US20230368472A1 (en) Dynamic augmented reality experience
US20220206582A1 (en) Media content items with haptic feedback augmentations
US20230289919A1 (en) Video stream refinement for dynamic scenes
KR20230160905A (ko) 햅틱 및 오디오 피드백 응답을 갖는 인터페이스
KR20230124081A (ko) 햅틱 피드백 응답을 갖는 통신 인터페이스
KR20230160918A (ko) 햅틱 및 오디오 피드백 응답을 갖는 인터페이스
CN113989463A (zh) 一种会议显示方法、装置、系统和电子设备
CN115861575A (zh) 商品虚拟试用效果展示方法及电子设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21804806

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2022567137

Country of ref document: JP

Kind code of ref document: A

ENP Entry into the national phase

Ref document number: 20227042575

Country of ref document: KR

Kind code of ref document: A

ENP Entry into the national phase

Ref document number: 2021804806

Country of ref document: EP

Effective date: 20221209

NENP Non-entry into the national phase

Ref country code: DE