WO2020147598A1 - Procédé et appareil d'action de modèle, haut-parleur comportant un écran, dispositif électronique et support de stockage - Google Patents

Procédé et appareil d'action de modèle, haut-parleur comportant un écran, dispositif électronique et support de stockage Download PDF

Info

Publication number
WO2020147598A1
WO2020147598A1 PCT/CN2020/070375 CN2020070375W WO2020147598A1 WO 2020147598 A1 WO2020147598 A1 WO 2020147598A1 CN 2020070375 W CN2020070375 W CN 2020070375W WO 2020147598 A1 WO2020147598 A1 WO 2020147598A1
Authority
WO
WIPO (PCT)
Prior art keywords
face
image
facial
preset
action
Prior art date
Application number
PCT/CN2020/070375
Other languages
English (en)
Chinese (zh)
Inventor
冯瑞丰
Original Assignee
北京字节跳动网络技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京字节跳动网络技术有限公司 filed Critical 北京字节跳动网络技术有限公司
Publication of WO2020147598A1 publication Critical patent/WO2020147598A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T13/00—Animation
    • G06T13/20—3D [Three Dimensional] animation
    • G06T13/40—3D [Three Dimensional] animation of characters, e.g. humans, animals or virtual beings
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/168—Feature extraction; Face representation

Definitions

  • the embodiments of the present disclosure relate to image processing technology, for example, to a model action method, device, speaker with screen, electronic equipment, and storage medium.
  • the human-machine interaction method stays in a two-dimensional space, such as using voice for interaction, or in screen interaction, by detecting the moving distance, moving speed and moving direction of the limbs on the plane to achieve the interaction.
  • the interaction in the two-dimensional space cannot simulate real characters, and the interaction effect is poor.
  • the present disclosure provides a model action method, a device, a speaker with a screen, an electronic device, and a storage medium, so as to solve the problem that the model action method in a two-dimensional space cannot simulate a real character image and the model action effect is poor.
  • the embodiment of the present disclosure provides a model action method, including:
  • a model action instruction is generated according to the facial action feature parameters, so that a preset three-dimensional (3 Dimensions, 3D) image with facial feature data executes a face action corresponding to the model action instruction according to the model action instruction.
  • the embodiment of the present disclosure also provides a model action device, which includes:
  • the face image acquisition module is set to acquire two or more consecutive face images
  • the facial motion feature parameter determination module is configured to determine the facial motion feature parameter corresponding to the face change according to the face changes in the two or more face images;
  • the preset 3D image action execution module is configured to generate model action instructions according to the facial action feature parameters, so that the preset 3D image with face feature data executes the person corresponding to the model action instruction according to the model action instructions Face action.
  • the embodiment of the present disclosure further provides a speaker with a screen, which includes a main body, a controller located in the main body, and at least two cameras located on the main body; the distance between the at least two cameras is greater than a distance threshold, and the controller
  • the model action device as described in any embodiment of the present disclosure is provided inside.
  • An embodiment of the present disclosure also provides an electronic device, which includes:
  • One or more processors are One or more processors;
  • Memory set to store one or more programs
  • the one or more processors When the one or more programs are executed by the one or more processors, the one or more processors implement the model action method described in any of the embodiments of the present disclosure.
  • the embodiment of the present disclosure also provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the model action method as described in any of the embodiments of the present disclosure is implemented.
  • FIG. 1 is a flowchart of a model action method provided by Embodiment 1 of the present disclosure
  • FIG. 2 is a flowchart of a model action method provided by Embodiment 2 of the present disclosure
  • FIG. 3 is a flowchart of a model action method provided by Embodiment 3 of the present disclosure.
  • Embodiment 4 is a schematic structural diagram of a model action device provided by Embodiment 4 of the present disclosure.
  • FIG. 5 is a schematic structural diagram of a speaker with a screen provided by Embodiment 5 of the present disclosure
  • FIG. 6 is a schematic diagram of the control structure of a model action interface of a speaker with a screen provided in Embodiment 5 of the present disclosure
  • FIG. 7 is a schematic structural diagram of an electronic device provided by Embodiment 6 of the present disclosure.
  • each embodiment provides optional features and examples at the same time. Multiple features recorded in the embodiments can be combined to form multiple alternative solutions. Each numbered embodiment should not be regarded as It is a technical solution.
  • Figure 1 is a flow chart of a model action method provided in Embodiment 1 of the present disclosure. This embodiment is applicable to the situation of human face action interaction.
  • the method can be executed by a model action device, which can be hardware and/or software. It can be integrated into electronic devices such as mobile phones, tablets and computers. Including the following steps:
  • Two or more consecutive face images are acquired through at least one camera, and the time interval between each image is preset.
  • a face image can be acquired by a camera at a preset time interval, or face video data can be recorded by a camera, and the face image can be intercepted from the face video data at a preset time interval.
  • the face changes in the face image can be detected.
  • the face change is determined by comparing the pixel changes at the corresponding positions in the two face images.
  • a fixed part of the face The face change determines the change of the face.
  • a fixed part can be the eyes or the chin.
  • the change of the eyes in the two images can have positions Change, according to the interval time between two images, the movement speed of the eyes can be confirmed.
  • the movement trajectory of the eyes can be obtained, and the movement speed information can be obtained by combining the movement time information, which can determine the person Feature parameters of facial movements.
  • the facial motion characteristic parameter includes at least one of the following parameters: moving speed, moving direction, and moving distance. Face movements can be shaking up and down, shaking left and right, or shaking in circles.
  • the facial motion characteristic parameter corresponding to the facial motion is at least one of the moving speed, the moving direction and the moving distance.
  • the facial motion characteristic parameters include movement speed, movement direction, and movement distance.
  • the facial motion feature parameters can be used to restore the facial motion to achieve the effect of interacting with the user.
  • the preset 3D image with facial feature data is: a cartoon 3D image, a professional 3D image, or a gender 3D image.
  • the cartoon 3D image can be, for example, an animal image such as a kitten, a puppy, or a monkey, or an animated image such as Peppa Pig or Cherry Ball.
  • the professional 3D image may be, for example, an image of a doctor, a teacher, a firefighter, or a policeman.
  • the gender 3D image can be, for example, a man or a woman.
  • the gender 3D image can also be combined with age information to set images such as boy, girl, adult man, adult woman, elderly man, or elderly woman.
  • the determining, based on the face changes in the two or more face images, the face action feature parameters corresponding to the face changes includes: according to the The position changes of the preset parts of the face in two or more face images are determined, and the moving speed, the moving direction and the moving distance of the preset parts are determined to determine the facial motion characteristic parameters.
  • the preset position may be, for example, the eyes or the chin, or the cheeks, and the preset position is not limited here.
  • the model action instruction is generated according to the facial action characteristic parameters, so that the preset 3D image with face feature data is executed according to the model action instruction corresponding to the model action instruction
  • the facial action includes: generating a model action instruction according to the moving speed, the moving direction, and the moving distance, so that a preset 3D image with facial feature data executes the model action according to the model action instruction Command the corresponding facial action.
  • the model action instructions include the movement speed, direction and distance of the face.
  • the preset 3D image with face feature data simulates the face action according to the model action instructions.
  • the preset 3D image with face feature data realizes the interaction with the user. Face action interaction.
  • the model action instruction includes the facial movement speed, movement direction and movement distance of the preset 3D image with facial feature data.
  • the facial movement speed, movement direction and movement distance of the preset 3D image with facial feature data can be compared with The moving speed, moving direction, and moving distance of the user's face are the same, or they can be set according to preset rules.
  • the preset rule can be, for example, that the moving speed of the user's face is v, which has facial feature data.
  • the moving speed of the face of the 3D image is 2v; the moving direction of the user’s face is left and right, and the moving direction of the preset 3D image with facial feature data is left and right; the moving distance of the user’s face is d, with people
  • the face movement distance of the preset 3D image of the facial feature data is 2d.
  • the preset rule can also be, for example, that the moving speed of the user's face is v, and the moving speed of the preset 3D image with facial feature data is v/2; the moving direction of the user's face is left and right, with people The moving direction of the face of the preset 3D image of the facial feature data is right and left; the moving distance of the user's face is d, and the moving distance of the face of the preset 3D image with facial feature data is d/2.
  • the preset rules can be set arbitrarily to improve the interactive interest.
  • the model action method obtained by the embodiment of the present disclosure obtains two or more consecutive face images; and determines the face change according to the face change in the two or more face images Corresponding face action feature parameters; generating model action instructions according to the face action feature parameters, so that the preset 3D image with face feature data executes the face actions corresponding to the model action instructions according to the model action instructions. Realize the simulation of facial motions through preset 3D images with facial feature data, improve the effect of facial motion simulation, enhance the reality of facial motion simulation, and improve the interactive experience.
  • FIG. 2 is a schematic flowchart of a model action method provided in Embodiment 2 of the disclosure. This embodiment is described on the basis of the optional solutions in the foregoing embodiment. It includes the following:
  • S220 According to the face changes in the two or more face images, determine the facial motion feature parameters corresponding to the face changes.
  • the facial feature data of the image can achieve the effect of improving the accuracy of the construction of the 3D face model.
  • the facial feature extraction algorithm the facial feature data of the face image is extracted.
  • the face feature data mainly represents the data of the eyes, eyebrows, nose, mouth, ears, and face contour, and the face can be uniquely represented by the face feature data.
  • a 3D face model corresponding to the face image is constructed.
  • the 3D model used may be a general face model or a three-dimensional deformed model.
  • S250 Apply the 3D face model to a preset 3D image, and obtain a preset 3D image with the face feature data.
  • the preset 3D image with facial feature data has the same facial feature data as the user, achieving the purpose of simulating the appearance of the user, and the preset 3D image can Realize the role-playing of the user.
  • the preset 3D image is a doctor, after applying the user's face 3D model to the preset 3D image, a doctor with the user's face data can be formed, that is, a doctor who looks the same as the user can be obtained , Realize the user to act as a doctor, improve entertainment.
  • the preset 3D image with facial feature data is a 3D image with user facial feature data.
  • Model action instructions are generated based on facial action feature parameters.
  • the preset 3D image with facial feature data executes the corresponding person according to the model action instructions.
  • Facial actions can realize facial action interactions between users and preset 3D images with facial feature data that have the same facial feature data as themselves, which enhances the interest and enhances the interactive experience.
  • the technical solution of this embodiment extracts face feature data of at least one face image at a preset angle by acquiring at least one face image at a preset angle; constructs at least one face image at a preset angle according to the face feature data Corresponding face 3D model; apply the face 3D model to the preset 3D image to obtain the preset 3D image with facial feature data, which can realize face action interaction with the avatar with the user's facial features, and improve entertainment Sex and enhance the interactive experience.
  • This embodiment does not limit the execution order of steps S210, S220 and steps S230, S240, and S250, and can be performed according to the order of this embodiment, or steps S230, S240, and S250 are performed first, and then steps S210 and S220 are performed. It may also be steps S210 and S220, which are executed synchronously with steps S230, S240 and S250.
  • FIG. 3 is a schematic flowchart of a model action method provided in Embodiment 3 of the disclosure. This embodiment is described on the basis of the optional solutions in the foregoing embodiment. It includes the following:
  • S320 Determine the moving speed, the moving direction and the moving distance of the preset part according to the position changes of the preset parts of the face in the two or more face images to determine the facial motion feature parameters.
  • the preset parts in the face image include at least: eyes, mouth, nose, eyebrows, chin, forehead and cheeks.
  • the position change of the mouth in the face image is detected.
  • the direction and distance of the movement of the mouth can be determined, and the time period corresponding to the position of the mouth can be determined through the time information of the face image. Then you can determine the speed of the mouth.
  • the moving speed, moving direction and moving distance of the mouth are the moving speed, moving direction and moving distance of the human face.
  • a model action instruction is generated.
  • the model action instruction contains the facial movement speed, movement direction and movement distance of the preset 3D image with facial feature data, and has facial features
  • the face moving speed, moving direction and moving distance of the preset 3D image of the data can be the same as or different from the user’s face moving speed, moving direction and moving distance. It can be set according to the preset rules, and the preset rules can be referred to The above disclosed embodiments.
  • the technical solution of this embodiment acquires two or more consecutive face images, and determines the preset according to the position change of the preset part of the face in the two or more face images
  • the movement speed, movement direction and movement distance of the parts are used to determine the facial motion characteristic parameters, and to improve the accuracy of the determination of the movement speed, movement direction and movement distance of the face; according to the movement speed, the movement direction and the movement distance
  • Generate a model action instruction to make a preset 3D image with facial feature data execute facial actions with the model action instruction according to the model action instruction, so as to realize the simulation of facial actions through the preset 3D image with facial feature data, Improve the effect of facial motion simulation, enhance the reality of facial motion simulation, and improve the interactive experience.
  • FIG. 4 is a schematic structural diagram of a model action device provided in the fourth embodiment of the disclosure.
  • the model action device includes: a face image acquisition module 410, a face action feature parameter determination module 420, and a preset 3D image action execution module 430, each of which is described below.
  • the face image acquisition module 410 is configured to acquire two or more consecutive face images.
  • the facial motion characteristic parameter determination module 420 is configured to determine the facial motion characteristic parameter corresponding to the facial change according to the facial changes in the two or more facial images.
  • the preset 3D image action execution module 430 is configured to generate model action instructions according to the facial action feature parameters, so that the preset 3D image with face feature data executes the model action instructions corresponding to the model action instructions according to the model action instructions. Face action.
  • the model action device provided in this embodiment can simulate a face action through a preset 3D image with face feature data, improve the effect of face action simulation, enhance the reality of face action simulation, and improve the interactive experience.
  • the device further includes: a preset 3D image acquisition module, configured to generate model action instructions according to the facial motion feature parameters, so that the preset 3D image with facial feature data is Before the model action instruction executes the face action corresponding to the model action instruction, acquiring at least one face image with a preset angle, and extracting face feature data of the face image at the at least one preset angle; Face feature data, constructing a face 3D model corresponding to the face image at the at least one preset angle; applying the face 3D model to an initial preset 3D image to obtain a preset with the face feature data 3D image.
  • a preset 3D image acquisition module configured to generate model action instructions according to the facial motion feature parameters, so that the preset 3D image with facial feature data is Before the model action instruction executes the face action corresponding to the model action instruction, acquiring at least one face image with a preset angle, and extracting face feature data of the face image at the at least one preset angle; Face feature data, constructing a face 3D model corresponding to the
  • the facial motion characteristic parameters include at least one of the following parameters: moving speed, moving direction, and moving distance.
  • the facial motion feature parameter determination module is set to:
  • the moving speed, the moving direction and the moving distance of the preset parts are determined to determine the facial motion characteristic parameters.
  • the preset 3D image action execution module is set to:
  • a model action instruction is generated according to the movement speed, the movement direction and the movement distance, so that the preset 3D avatar with facial feature data executes the face action corresponding to the model action instruction according to the model action instruction.
  • the preset 3D image with facial feature data is: a cartoon 3D image, a professional 3D image, or a gender 3D image.
  • the model action device provided by the present disclosure can execute the model action method provided by any embodiment of the present disclosure, and has the corresponding functional modules and effects for executing the model action method.
  • Fig. 5 is a schematic structural diagram of a speaker with a screen provided in the fifth embodiment of the disclosure.
  • the speaker with a screen includes: a main body 51, a controller 52 located in the main body 51, and at least two cameras 53 located on the main body 51; the distance between the at least two cameras 53 is greater than the distance threshold, the The controller 52 is provided with any model action device as provided in the embodiment of the present disclosure.
  • the distance between at least two cameras is greater than the distance threshold.
  • one camera can be placed on the upper part of the main body of the speaker with screen, and the other camera can be placed on the lower part of the main body of the speaker with screen, and the distance is greater than the distance threshold . It is convenient to shoot face images from multiple angles and directions, enrich the angle information obtained by the face image, and then enrich the face feature data of the face image, so as to achieve the effect of improving the accuracy of the construction of the face 3D model.
  • Figure 6 is a schematic diagram of the control structure of a model action interface with a speaker with a screen provided by an embodiment of the disclosure.
  • the camera obtains the user's face image data, extracts the face feature data, builds a face 3D model, and applies the face 3D model to a preset 3D image to obtain a preset 3D with face feature data Image, so that the preset 3D image with facial feature data has the user's facial feature data, and then long press control 3, the camera captures the user's facial movements, the preset 3D image with facial feature data is based on the user's face
  • the moving speed, moving direction and moving distance are executed in accordance with the conversion rules set by the user to perform facial actions to realize interaction with the user through facial actions.
  • a speaker with a screen can be applied to a point-to-read scene.
  • the image data of the point-to-read material is obtained through a camera of the screen-mounted speaker.
  • the image text content is obtained, the text content is converted into voice data, and the reading is realized through the speaker.
  • the screen in the speaker with screen can display the preset 3D image with face feature data, and click the preset 3D image with face feature data to realize the interesting point of reading, such as the preset with face feature data
  • the 3D image is the teacher, through the teacher’s point reading, the simulation of real teaching is realized, the learning fun is improved, and the learning efficiency is improved.
  • the user can also use the preset 3D image with facial feature data displayed in the speaker with a screen, and use the model action method in the above disclosed embodiment to realize the interaction with the preset 3D image with facial feature data to improve the interaction Experience.
  • the speaker with a screen provided in this embodiment can simulate facial motions through a preset 3D image with facial feature data, improve the effect of facial motion simulation, enhance the authenticity of facial motion simulation, and improve interactive experience.
  • FIG. 7 shows a schematic structural diagram of an electronic device 600 suitable for implementing embodiments of the present disclosure.
  • the electronic devices in the embodiments of the present disclosure may include, but are not limited to, mobile phones, notebook computers, digital broadcast receivers, personal digital assistants (Personal Digital Assistant, PDA), tablet computers (PAD), and portable multimedia players (Portable Media Player). , PMP), mobile terminals such as car navigation terminals, etc., and fixed terminals such as digital televisions (Television, TV), desktop computers, etc.
  • PMP Personal Digital Assistant
  • PMP Personal Digital Assistant
  • mobile terminals such as car navigation terminals, etc.
  • fixed terminals such as digital televisions (Television, TV), desktop computers, etc.
  • the electronic device shown in FIG. 7 is only an example, and should not bring any limitation to the function and scope of use of the embodiments of the present disclosure.
  • the electronic device 600 may include a processing device (such as a central processing unit, a graphics processor, etc.) 601, which may be based on a program stored in a read-only memory (Read-Only Memory, ROM) 602 or from a storage device 608 is loaded into a random access memory (Random Access Memory, RAM) 603 program to perform various appropriate actions and processing.
  • the RAM 603 also stores various programs and data required for the operation of the electronic device 600.
  • the processing device 601, the ROM 602, and the RAM 603 are connected to each other through a bus 604.
  • An input/output (Input/Output, I/O) interface 605 is also connected to the bus 604.
  • the following devices can be connected to the I/O interface 605: including input devices 606 such as touch screen, touch panel, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; including, for example, a liquid crystal display (LCD) Output devices 607 such as speakers, vibrators, etc.; storage devices 608 such as magnetic tapes, hard disks, etc.; and communication devices 609.
  • the communication device 609 may allow the electronic device 600 to perform wireless or wired communication with other devices to exchange data.
  • FIG. 7 shows an electronic device 600 having multiple devices, it is not required to implement or have all the illustrated devices. More or fewer devices may be implemented or provided instead.
  • the process described above with reference to the flowchart may be implemented as a computer software program.
  • the embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart.
  • the computer program may be downloaded and installed from the network through the communication device 609, or from the storage device 608, or from the ROM 602.
  • the processing device 601 the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.
  • the aforementioned computer-readable medium of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the two.
  • the computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination of the above.
  • Examples of computer-readable storage media may include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, RAM, ROM, Erasable Programmable Read-Only Memory (EPROM) Or flash memory), optical fiber, CD-ROM (Compact Disc Read-Only Memory), optical storage device, magnetic storage device, or any suitable combination of the above.
  • the computer-readable storage medium may be any tangible medium that contains or stores a program, and the program may be used by or in combination with an instruction execution system, apparatus, or device.
  • the computer-readable signal medium may include a data signal that is propagated in baseband or as part of a carrier wave, in which computer-readable program code is carried. This propagated data signal can take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing.
  • the computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, and the computer-readable signal medium may send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device .
  • the program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: electric wires, optical cables, radio frequency (RF), etc., or any suitable combination of the foregoing.
  • the above-mentioned computer-readable medium may be included in the above-mentioned electronic device; or it may exist alone without being assembled into the electronic device.
  • the above-mentioned computer-readable medium carries one or more programs, and when the above-mentioned one or more programs are executed by the electronic device, the electronic device: acquires two or more consecutive face images; The face changes in two or more face images are determined to determine the face action feature parameters corresponding to the face changes; the model action instructions are generated according to the face action feature parameters, so as to have the face feature data. It is assumed that the 3D avatar executes the facial action corresponding to the model action instruction according to the model action instruction.
  • the computer program code for performing the operations of the present disclosure can be written in one or more programming languages or a combination thereof.
  • the above programming languages include object-oriented programming languages such as Java, Smalltalk, C++, as well as conventional Procedural programming language-such as "C" language or similar programming language.
  • the program code can be executed entirely on the user's computer, partly on the user's computer, executed as an independent software package, partly on the user's computer and partly executed on a remote computer, or entirely executed on the remote computer or server.
  • the remote computer can be connected to the user's computer through any kind of network-including Local Area Network (LAN) or Wide Area Network (WAN)-or it can be connected to an external computer ( For example, use an Internet service provider to connect via the Internet).
  • LAN Local Area Network
  • WAN Wide Area Network
  • each block in the flowchart or block diagram may represent a module, program segment, or part of code that contains one or more logic functions Executable instructions.
  • the functions marked in the block may also occur in a different order from the order marked in the drawings. For example, two blocks shown in succession can actually be executed substantially in parallel, or they can sometimes be executed in the reverse order, depending on the functions involved.
  • each block in the block diagrams and/or flowcharts, and combinations of blocks in the block diagrams and/or flowcharts can be implemented with dedicated hardware-based systems that perform specified functions or operations Or, it can be realized by a combination of dedicated hardware and computer instructions.
  • the units described in the embodiments of the present disclosure may be implemented in software or hardware. Among them, the name of the unit in one case does not constitute a limitation on the unit itself.
  • the preset 3D image action execution module can also be described as an "action execution module”.
  • the seventh embodiment of the present disclosure also provides a computer-readable storage medium on which a computer program is stored.
  • the model action method as provided in the embodiment of the present disclosure is realized, and the method includes: obtaining continuous Two or more face images; according to the face changes in the two or more face images, determine the face action feature parameters corresponding to the face changes; according to the face action
  • the feature parameter generates a model action instruction, so that the preset 3D avatar with facial feature data executes the face action corresponding to the model action instruction according to the model action instruction.
  • the computer-readable storage medium provided by the embodiment of the present disclosure is not limited to the implementation of the above method operation when the computer program stored thereon is executed, and can also implement the model action method provided in any embodiment of the present disclosure. Related operations.
  • the present disclosure can be implemented by software and necessary general-purpose hardware, and of course, it can also be implemented by hardware. Based on this understanding, the technical solution of the present disclosure can be embodied in the form of a software product.
  • the computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, ROM, RAM, flash memory (FLASH), hard disk, or optical disk. Etc., including multiple instructions to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute the method described in the embodiments of the present disclosure.
  • the included units and modules are only divided according to the functional logic, but are not limited to the above division, as long as the corresponding functions can be realized; in addition, the names of the functional units are only for It is easy to distinguish each other and is not used to limit the protection scope of the present disclosure.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Health & Medical Sciences (AREA)
  • Human Computer Interaction (AREA)
  • Multimedia (AREA)
  • Processing Or Creating Images (AREA)

Abstract

La présente invention concerne un procédé et un appareil d'action de modèle, un haut-parleur comportant un écran, un dispositif électronique et un support de stockage. Ledit procédé consiste à : acquérir au moins deux images faciales continues (S110) ; déterminer, en fonction d'un changement facial dans lesdites au moins deux images faciales, des paramètres de caractéristique d'action faciale correspondant au changement facial (S120) ; et générer une instruction d'action de modèle selon les paramètres de caractéristique d'action faciale, de façon à permettre à une image 3D prédéfinie ayant des données de caractéristique faciale d'exécuter, conformément à l'instruction d'action de modèle, une action faciale correspondant à l'instruction d'action de modèle (S130).
PCT/CN2020/070375 2019-01-15 2020-01-06 Procédé et appareil d'action de modèle, haut-parleur comportant un écran, dispositif électronique et support de stockage WO2020147598A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910037303.3A CN111435546A (zh) 2019-01-15 2019-01-15 模型动作方法、装置、带屏音箱、电子设备及存储介质
CN201910037303.3 2019-01-15

Publications (1)

Publication Number Publication Date
WO2020147598A1 true WO2020147598A1 (fr) 2020-07-23

Family

ID=71580067

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/070375 WO2020147598A1 (fr) 2019-01-15 2020-01-06 Procédé et appareil d'action de modèle, haut-parleur comportant un écran, dispositif électronique et support de stockage

Country Status (2)

Country Link
CN (1) CN111435546A (fr)
WO (1) WO2020147598A1 (fr)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103105924A (zh) * 2011-11-15 2013-05-15 中国科学院深圳先进技术研究院 人机交互方法和装置
CN104616347A (zh) * 2015-01-05 2015-05-13 掌赢信息科技（上海）有限公司 一种表情迁移方法、电子设备及系统
US20150178988A1 (en) * 2012-05-22 2015-06-25 Telefonica, S.A. Method and a system for generating a realistic 3d reconstruction model for an object or being
CN106447785A (zh) * 2016-09-30 2017-02-22 北京奇虎科技有限公司 一种驱动虚拟角色的方法和装置
CN107479693A (zh) * 2017-07-07 2017-12-15 大圣科技股份有限公司 基于rgb信息的实时手部识别方法、存储介质、电子设备
CN108875633A (zh) * 2018-06-19 2018-11-23 北京旷视科技有限公司 表情检测与表情驱动方法、装置和系统及存储介质

Family Cites Families (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1991817A (zh) * 2005-12-29 2007-07-04 罗兆鑫 电子邮件辅助暨文字转语音系统
CN102169642B (zh) * 2011-04-06 2013-04-03 沈阳航空航天大学 具有智能纠错功能的交互式虚拟教师系统
CN102222363B (zh) * 2011-07-19 2012-10-03 杭州实时数码科技有限公司 从人脸图像快速构建高精度个性化人脸模型的方法
CN103414782A (zh) * 2013-08-20 2013-11-27 苏州跨界软件科技有限公司 一种基于虚拟人物的亲子系统及方法
CN103413468A (zh) * 2013-08-20 2013-11-27 苏州跨界软件科技有限公司 一种基于虚拟人物的亲子教育方法
CN105590486A (zh) * 2014-10-21 2016-05-18 黄小曼 一种基于机器视觉的座台式点读机及相关系统装置与方法
CN105518582B (zh) * 2015-06-30 2018-02-02 北京旷视科技有限公司 活体检测方法及设备
CN107333086A (zh) * 2016-04-29 2017-11-07 掌赢信息科技（上海）有限公司 一种在虚拟场景中进行视频通信的方法及装置
CN106023692A (zh) * 2016-05-13 2016-10-12 广东博士早教科技有限公司 一种基于娱乐交互的ar趣味学习系统及方法
CN108229239B (zh) * 2016-12-09 2020-07-10 武汉斗鱼网络科技有限公司 一种图像处理的方法及装置
CN106910247B (zh) * 2017-03-20 2020-10-02 厦门黑镜科技有限公司 用于生成三维头像模型的方法和装置
CN107705355A (zh) * 2017-09-08 2018-02-16 郭睿 一种基于多张图片的3d人体建模方法及装置
CN107831902B (zh) * 2017-11-23 2020-08-25 腾讯科技（上海）有限公司 一种运动控制方法及其设备、存储介质、终端
CN108090463B (zh) * 2017-12-29 2021-10-26 腾讯科技（深圳）有限公司 对象控制方法、装置、存储介质和计算机设备
CN108615256B (zh) * 2018-03-29 2022-04-12 西南民族大学 一种人脸三维重建方法及装置
CN108806360A (zh) * 2018-05-31 2018-11-13 北京智能管家科技有限公司 伴读方法、装置、设备和存储介质
CN109118562A (zh) * 2018-08-31 2019-01-01 百度在线网络技术（北京）有限公司 虚拟形象的讲解视频制作方法、装置以及终端

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103105924A (zh) * 2011-11-15 2013-05-15 中国科学院深圳先进技术研究院 人机交互方法和装置
US20150178988A1 (en) * 2012-05-22 2015-06-25 Telefonica, S.A. Method and a system for generating a realistic 3d reconstruction model for an object or being
CN104616347A (zh) * 2015-01-05 2015-05-13 掌赢信息科技（上海）有限公司 一种表情迁移方法、电子设备及系统
CN106447785A (zh) * 2016-09-30 2017-02-22 北京奇虎科技有限公司 一种驱动虚拟角色的方法和装置
CN107479693A (zh) * 2017-07-07 2017-12-15 大圣科技股份有限公司 基于rgb信息的实时手部识别方法、存储介质、电子设备
CN108875633A (zh) * 2018-06-19 2018-11-23 北京旷视科技有限公司 表情检测与表情驱动方法、装置和系统及存储介质

Also Published As

Publication number Publication date
CN111435546A (zh) 2020-07-21

Similar Documents

Publication Publication Date Title
US20210029305A1 (en) Method and apparatus for adding a video special effect, terminal device and storage medium
CN112379812B (zh) 仿真3d数字人交互方法、装置、电子设备及存储介质
CN109462776B (zh) 一种视频特效添加方法、装置、终端设备及存储介质
WO2020186935A1 (fr) Procédé et dispositif d'affichage d'objet virtuel, appareil électronique, et support de stockage lisible par ordinateur
WO2022068479A1 (fr) Procédé et appareil de traitement d'image, ainsi que dispositif électronique et support de stockage lisible par ordinateur
WO2020107908A1 (fr) Procédé et appareil d'ajout d'effets spéciaux vidéo associés à de multiples utilisateurs, dispositif terminal et support de stockage
WO2022116751A1 (fr) Procédé et appareil d'interaction, ainsi que terminal, serveur et support de stockage
US10166477B2 (en) Image processing device, image processing method, and image processing program
CN109474850B (zh) 运动像素视频特效添加方法、装置、终端设备及存储介质
US20230419582A1 (en) Virtual object display method and apparatus, electronic device, and medium
WO2022088928A1 (fr) Procédé et appareil de rendu d'objet élastique, dispositif et support de stockage
WO2022170958A1 (fr) Procédé et dispositif d'affichage basés sur une réalité augmentée, support de stockage et produit-programme
WO2020186934A1 (fr) Procédé, appareil et dispositif électronique pour générer un arrière-plan dynamique contenant une animation
CN109600559B (zh) 一种视频特效添加方法、装置、终端设备及存储介质
WO2023116653A1 (fr) Procédé et appareil d'affichage d'élément, et dispositif électronique et support de stockage
TW200541330A (en) Method and system for real-time interactive video
US20230182028A1 (en) Game live broadcast interaction method and apparatus
WO2022055421A1 (fr) Procédé d'affichage fondé sur la réalité augmentée, dispositif, et support de stockage
JP2022500795A (ja) アバターアニメーション
WO2022012349A1 (fr) Procédé et appareil de traitement d'animation, dispositif électronique et support de stockage
US11756251B2 (en) Facial animation control by automatic generation of facial action units using text and speech
WO2023116562A1 (fr) Procédé et appareil d'affichage d'image, dispositif électronique et support de stockage
WO2020147598A1 (fr) Procédé et appareil d'action de modèle, haut-parleur comportant un écran, dispositif électronique et support de stockage
WO2023195909A2 (fr) Procédé et appareil de détermination de vidéo à effets spéciaux, dispositif électronique et support de stockage
EP4071725A1 (fr) Procédé et dispositif d'affichage basés sur une réalité augmentée, support de stockage et produit-programme

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20741152

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 05.11.2021)

122 Ep: pct application non-entry in european phase

Ref document number: 20741152

Country of ref document: EP

Kind code of ref document: A1