WO2020147598A1

WO2020147598A1 - Model action method and apparatus, speaker having screen, electronic device, and storage medium

Info

Publication number: WO2020147598A1
Application number: PCT/CN2020/070375
Authority: WO
Inventors: 冯瑞丰
Original assignee: 北京字节跳动网络技术有限公司
Priority date: 2019-01-15
Filing date: 2020-01-06
Publication date: 2020-07-23
Also published as: CN111435546A

Abstract

A model action method and apparatus, a speaker having a screen, an electronic device, and a storage medium. Said method comprises: acquiring two or more continuous facial images (S110); determining, according to a facial change in the two or more facial images, facial action feature parameters corresponding to the facial change (S120); and generating a model action instruction according to the facial action feature parameters, so as to enable a pre-set 3D image having facial feature data to execute, according to the model action instruction, a facial action corresponding to the model action instruction (S130).

Description

Model action method, device, speaker with screen, electronic equipment and storage medium

This application claims the priority of a Chinese patent application filed with the Chinese Patent Office with an application number of 201910037303.3 on January 15, 2019. The entire content of the application is incorporated into this application by reference.

Technical field

The embodiments of the present disclosure relate to image processing technology, for example, to a model action method, device, speaker with screen, electronic equipment, and storage medium.

Background technique

With the development of electronic products, good interaction with electronic products can meet users' needs and improve users' experience of using electronic products.

The human-machine interaction method stays in a two-dimensional space, such as using voice for interaction, or in screen interaction, by detecting the moving distance, moving speed and moving direction of the limbs on the plane to achieve the interaction. The interaction in the two-dimensional space cannot simulate real characters, and the interaction effect is poor.

Summary of the invention

The present disclosure provides a model action method, a device, a speaker with a screen, an electronic device, and a storage medium, so as to solve the problem that the model action method in a two-dimensional space cannot simulate a real character image and the model action effect is poor.

The embodiment of the present disclosure provides a model action method, including:

Acquire two or more consecutive face images;

Determining the facial motion feature parameters corresponding to the facial changes according to the facial changes in the two or more facial images;

A model action instruction is generated according to the facial action feature parameters, so that a preset three-dimensional (3 Dimensions, 3D) image with facial feature data executes a face action corresponding to the model action instruction according to the model action instruction.

The embodiment of the present disclosure also provides a model action device, which includes:

The face image acquisition module is set to acquire two or more consecutive face images;

The facial motion feature parameter determination module is configured to determine the facial motion feature parameter corresponding to the face change according to the face changes in the two or more face images;

The preset 3D image action execution module is configured to generate model action instructions according to the facial action feature parameters, so that the preset 3D image with face feature data executes the person corresponding to the model action instruction according to the model action instructions Face action.

The embodiment of the present disclosure further provides a speaker with a screen, which includes a main body, a controller located in the main body, and at least two cameras located on the main body; the distance between the at least two cameras is greater than a distance threshold, and the controller The model action device as described in any embodiment of the present disclosure is provided inside.

An embodiment of the present disclosure also provides an electronic device, which includes:

One or more processors;

Memory, set to store one or more programs;

When the one or more programs are executed by the one or more processors, the one or more processors implement the model action method described in any of the embodiments of the present disclosure.

The embodiment of the present disclosure also provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the model action method as described in any of the embodiments of the present disclosure is implemented.

BRIEF DESCRIPTION

FIG. 1 is a flowchart of a model action method provided by Embodiment 1 of the present disclosure;

FIG. 2 is a flowchart of a model action method provided by Embodiment 2 of the present disclosure;

FIG. 3 is a flowchart of a model action method provided by Embodiment 3 of the present disclosure;

4 is a schematic structural diagram of a model action device provided by Embodiment 4 of the present disclosure;

FIG. 5 is a schematic structural diagram of a speaker with a screen provided by Embodiment 5 of the present disclosure;

6 is a schematic diagram of the control structure of a model action interface of a speaker with a screen provided in Embodiment 5 of the present disclosure;

FIG. 7 is a schematic structural diagram of an electronic device provided by Embodiment 6 of the present disclosure.

detailed description

The present disclosure will be described below with reference to the drawings and embodiments. The specific embodiments described here are only used to explain the present disclosure, but not to limit the present disclosure. For the convenience of description, only a part of the structure related to the present disclosure is shown in the drawings instead of all of the structure.

In the following embodiments, each embodiment provides optional features and examples at the same time. Multiple features recorded in the embodiments can be combined to form multiple alternative solutions. Each numbered embodiment should not be regarded as It is a technical solution.

Example one

Figure 1 is a flow chart of a model action method provided in Embodiment 1 of the present disclosure. This embodiment is applicable to the situation of human face action interaction. The method can be executed by a model action device, which can be hardware and/or software. It can be integrated into electronic devices such as mobile phones, tablets and computers. Including the following steps:

S110. Acquire two or more consecutive face images.

Two or more consecutive face images are acquired through at least one camera, and the time interval between each image is preset. Optionally, a face image can be acquired by a camera at a preset time interval, or face video data can be recorded by a camera, and the face image can be intercepted from the face video data at a preset time interval.

S120: According to the face changes in the two or more face images, determine the facial motion feature parameters corresponding to the face changes.

Through two or more face images, the face changes in the face image can be detected. In one embodiment, when detecting a face change in a face image, the face change is determined by comparing the pixel changes at the corresponding positions in the two face images. For example, a fixed part of the face The face change determines the change of the face. A fixed part can be the eyes or the chin. Take the eyes as an example. Take the change of the eyes in the face in two images to determine the change of the face. The change of the eyes in the two images can have positions Change, according to the interval time between two images, the movement speed of the eyes can be confirmed. Through the change of the position of the eyes in multiple images, the movement trajectory of the eyes can be obtained, and the movement speed information can be obtained by combining the movement time information, which can determine the person Feature parameters of facial movements.

Optionally, the facial motion characteristic parameter includes at least one of the following parameters: moving speed, moving direction, and moving distance. Face movements can be shaking up and down, shaking left and right, or shaking in circles. The facial motion characteristic parameter corresponding to the facial motion is at least one of the moving speed, the moving direction and the moving distance. Optionally, the facial motion characteristic parameters include movement speed, movement direction, and movement distance. The facial motion feature parameters can be used to restore the facial motion to achieve the effect of interacting with the user.

S130. Generate a model action instruction according to the face action feature parameters, and make a preset 3D image with face feature data execute a face action corresponding to the model action instruction according to the model action instruction.

In this embodiment, the preset 3D image with facial feature data is: a cartoon 3D image, a professional 3D image, or a gender 3D image. The cartoon 3D image can be, for example, an animal image such as a kitten, a puppy, or a monkey, or an animated image such as Peppa Pig or Cherry Ball. The professional 3D image may be, for example, an image of a doctor, a teacher, a firefighter, or a policeman. The gender 3D image can be, for example, a man or a woman. The gender 3D image can also be combined with age information to set images such as boy, girl, adult man, adult woman, elderly man, or elderly woman.

In the above solution, optionally, the determining, based on the face changes in the two or more face images, the face action feature parameters corresponding to the face changes includes: according to the The position changes of the preset parts of the face in two or more face images are determined, and the moving speed, the moving direction and the moving distance of the preset parts are determined to determine the facial motion characteristic parameters. Wherein, the preset position may be, for example, the eyes or the chin, or the cheeks, and the preset position is not limited here.

In the above solution, optionally, the model action instruction is generated according to the facial action characteristic parameters, so that the preset 3D image with face feature data is executed according to the model action instruction corresponding to the model action instruction The facial action includes: generating a model action instruction according to the moving speed, the moving direction, and the moving distance, so that a preset 3D image with facial feature data executes the model action according to the model action instruction Command the corresponding facial action. The model action instructions include the movement speed, direction and distance of the face. The preset 3D image with face feature data simulates the face action according to the model action instructions. The preset 3D image with face feature data realizes the interaction with the user. Face action interaction. The model action instruction includes the facial movement speed, movement direction and movement distance of the preset 3D image with facial feature data. The facial movement speed, movement direction and movement distance of the preset 3D image with facial feature data can be compared with The moving speed, moving direction, and moving distance of the user's face are the same, or they can be set according to preset rules. The preset rule can be, for example, that the moving speed of the user's face is v, which has facial feature data. The moving speed of the face of the 3D image is 2v; the moving direction of the user’s face is left and right, and the moving direction of the preset 3D image with facial feature data is left and right; the moving distance of the user’s face is d, with people The face movement distance of the preset 3D image of the facial feature data is 2d. The preset rule can also be, for example, that the moving speed of the user's face is v, and the moving speed of the preset 3D image with facial feature data is v/2; the moving direction of the user's face is left and right, with people The moving direction of the face of the preset 3D image of the facial feature data is right and left; the moving distance of the user's face is d, and the moving distance of the face of the preset 3D image with facial feature data is d/2. The preset rules can be set arbitrarily to improve the interactive interest.

The model action method provided by the embodiment of the present disclosure obtains two or more consecutive face images; and determines the face change according to the face change in the two or more face images Corresponding face action feature parameters; generating model action instructions according to the face action feature parameters, so that the preset 3D image with face feature data executes the face actions corresponding to the model action instructions according to the model action instructions. Realize the simulation of facial motions through preset 3D images with facial feature data, improve the effect of facial motion simulation, enhance the reality of facial motion simulation, and improve the interactive experience.

Example 2

FIG. 2 is a schematic flowchart of a model action method provided in Embodiment 2 of the disclosure. This embodiment is described on the basis of the optional solutions in the foregoing embodiment. It includes the following:

S210. Acquire two or more consecutive face images.

S220: According to the face changes in the two or more face images, determine the facial motion feature parameters corresponding to the face changes.

S230. Acquire at least one face image of a preset angle, and extract face feature data of the face image of the at least one preset angle.

Obtain at least one face image with a preset angle through at least two cameras. The distance between the at least two cameras is greater than the distance threshold, which can facilitate the shooting of face images from multiple angles and directions, enrich the angle information obtained from the face image, and then enrich the face The facial feature data of the image can achieve the effect of improving the accuracy of the construction of the 3D face model. Through the facial feature extraction algorithm, the facial feature data of the face image is extracted. The face feature data mainly represents the data of the eyes, eyebrows, nose, mouth, ears, and face contour, and the face can be uniquely represented by the face feature data.

S240. Construct a 3D face model corresponding to the face image of the at least one preset angle according to the face feature data.

According to the acquired facial feature data, a 3D face model corresponding to the face image is constructed. When constructing the face 3D model, the 3D model used may be a general face model or a three-dimensional deformed model.

S250: Apply the 3D face model to a preset 3D image, and obtain a preset 3D image with the face feature data.

Applying the constructed 3D face model to a preset 3D image, the preset 3D image with facial feature data has the same facial feature data as the user, achieving the purpose of simulating the appearance of the user, and the preset 3D image can Realize the role-playing of the user. If the preset 3D image is a doctor, after applying the user's face 3D model to the preset 3D image, a doctor with the user's face data can be formed, that is, a doctor who looks the same as the user can be obtained , Realize the user to act as a doctor, improve entertainment.

S260. Generate a model action instruction according to the facial action feature parameters, and make a preset 3D image with face feature data execute a face action corresponding to the model action instruction according to the model action instruction.

The preset 3D image with facial feature data is a 3D image with user facial feature data. Model action instructions are generated based on facial action feature parameters. The preset 3D image with facial feature data executes the corresponding person according to the model action instructions. Facial actions can realize facial action interactions between users and preset 3D images with facial feature data that have the same facial feature data as themselves, which enhances the interest and enhances the interactive experience.

The technical solution of this embodiment extracts face feature data of at least one face image at a preset angle by acquiring at least one face image at a preset angle; constructs at least one face image at a preset angle according to the face feature data Corresponding face 3D model; apply the face 3D model to the preset 3D image to obtain the preset 3D image with facial feature data, which can realize face action interaction with the avatar with the user's facial features, and improve entertainment Sex and enhance the interactive experience.

This embodiment does not limit the execution order of steps S210, S220 and steps S230, S240, and S250, and can be performed according to the order of this embodiment, or steps S230, S240, and S250 are performed first, and then steps S210 and S220 are performed. It may also be steps S210 and S220, which are executed synchronously with steps S230, S240 and S250.

Example three

FIG. 3 is a schematic flowchart of a model action method provided in Embodiment 3 of the disclosure. This embodiment is described on the basis of the optional solutions in the foregoing embodiment. It includes the following:

S310. Acquire two or more consecutive face images.

S320: Determine the moving speed, the moving direction and the moving distance of the preset part according to the position changes of the preset parts of the face in the two or more face images to determine the facial motion feature parameters.

The preset parts in the face image include at least: eyes, mouth, nose, eyebrows, chin, forehead and cheeks. In this embodiment, taking the preset part in the face image as the mouth as an example, the position change of the mouth in the face image is detected. In the acquired multiple face images, by detecting the position of the mouth in the face image, the direction and distance of the movement of the mouth can be determined, and the time period corresponding to the position of the mouth can be determined through the time information of the face image. Then you can determine the speed of the mouth. The moving speed, moving direction and moving distance of the mouth are the moving speed, moving direction and moving distance of the human face.

S330. Generate a model action instruction according to the movement speed, the movement direction, and the movement distance, so that a preset 3D image with facial feature data executes a face corresponding to the model action instruction according to the model action instruction action.

According to the user's facial movement speed, movement direction and movement distance, a model action instruction is generated. The model action instruction contains the facial movement speed, movement direction and movement distance of the preset 3D image with facial feature data, and has facial features The face moving speed, moving direction and moving distance of the preset 3D image of the data can be the same as or different from the user’s face moving speed, moving direction and moving distance. It can be set according to the preset rules, and the preset rules can be referred to The above disclosed embodiments.

The technical solution of this embodiment acquires two or more consecutive face images, and determines the preset according to the position change of the preset part of the face in the two or more face images The movement speed, movement direction and movement distance of the parts are used to determine the facial motion characteristic parameters, and to improve the accuracy of the determination of the movement speed, movement direction and movement distance of the face; according to the movement speed, the movement direction and the movement distance Generate a model action instruction to make a preset 3D image with facial feature data execute facial actions with the model action instruction according to the model action instruction, so as to realize the simulation of facial actions through the preset 3D image with facial feature data, Improve the effect of facial motion simulation, enhance the reality of facial motion simulation, and improve the interactive experience.

Example 4

FIG. 4 is a schematic structural diagram of a model action device provided in the fourth embodiment of the disclosure. Referring to FIG. 4, the model action device includes: a face image acquisition module 410, a face action feature parameter determination module 420, and a preset 3D image action execution module 430, each of which is described below.

The face image acquisition module 410 is configured to acquire two or more consecutive face images.

The facial motion characteristic parameter determination module 420 is configured to determine the facial motion characteristic parameter corresponding to the facial change according to the facial changes in the two or more facial images.

The preset 3D image action execution module 430 is configured to generate model action instructions according to the facial action feature parameters, so that the preset 3D image with face feature data executes the model action instructions corresponding to the model action instructions according to the model action instructions. Face action.

The model action device provided in this embodiment can simulate a face action through a preset 3D image with face feature data, improve the effect of face action simulation, enhance the reality of face action simulation, and improve the interactive experience.

In the above solution, optionally, the device further includes: a preset 3D image acquisition module, configured to generate model action instructions according to the facial motion feature parameters, so that the preset 3D image with facial feature data is Before the model action instruction executes the face action corresponding to the model action instruction, acquiring at least one face image with a preset angle, and extracting face feature data of the face image at the at least one preset angle; Face feature data, constructing a face 3D model corresponding to the face image at the at least one preset angle; applying the face 3D model to an initial preset 3D image to obtain a preset with the face feature data 3D image.

In an embodiment, the facial motion characteristic parameters include at least one of the following parameters: moving speed, moving direction, and moving distance.

In the above solution, optionally, the facial motion feature parameter determination module is set to:

According to the position changes of the preset parts of the face in the two or more face images, the moving speed, the moving direction and the moving distance of the preset parts are determined to determine the facial motion characteristic parameters.

In the above solution, it is optional that the preset 3D image action execution module is set to:

A model action instruction is generated according to the movement speed, the movement direction and the movement distance, so that the preset 3D avatar with facial feature data executes the face action corresponding to the model action instruction according to the model action instruction.

In an embodiment, the preset 3D image with facial feature data is: a cartoon 3D image, a professional 3D image, or a gender 3D image.

The model action device provided by the present disclosure can execute the model action method provided by any embodiment of the present disclosure, and has the corresponding functional modules and effects for executing the model action method.

Example 5

Fig. 5 is a schematic structural diagram of a speaker with a screen provided in the fifth embodiment of the disclosure. 5, the speaker with a screen includes: a main body 51, a controller 52 located in the main body 51, and at least two cameras 53 located on the main body 51; the distance between the at least two cameras 53 is greater than the distance threshold, the The controller 52 is provided with any model action device as provided in the embodiment of the present disclosure.

The distance between at least two cameras is greater than the distance threshold. Taking two cameras as an example, one camera can be placed on the upper part of the main body of the speaker with screen, and the other camera can be placed on the lower part of the main body of the speaker with screen, and the distance is greater than the distance threshold , It is convenient to shoot face images from multiple angles and directions, enrich the angle information obtained by the face image, and then enrich the face feature data of the face image, so as to achieve the effect of improving the accuracy of the construction of the face 3D model.

Figure 6 is a schematic diagram of the control structure of a model action interface with a speaker with a screen provided by an embodiment of the disclosure. After the user starts the model action interface, the user selects the image type of the preset 3D image with facial feature data through the setting menu control 1. The conversion rule between the face movement speed, movement direction and movement distance of the preset 3D image with facial feature data and the movement speed, movement direction and movement distance of the user's face. In the preset 3D image type setting with facial feature data, you can quickly set it through control 2. First, click Control 3 many times, the camera obtains the user's face image data, extracts the face feature data, builds a face 3D model, and applies the face 3D model to a preset 3D image to obtain a preset 3D with face feature data Image, so that the preset 3D image with facial feature data has the user's facial feature data, and then long press control 3, the camera captures the user's facial movements, the preset 3D image with facial feature data is based on the user's face The moving speed, moving direction and moving distance are executed in accordance with the conversion rules set by the user to perform facial actions to realize interaction with the user through facial actions.

Exemplarily, a speaker with a screen can be applied to a point-to-read scene. The image data of the point-to-read material is obtained through a camera of the screen-mounted speaker. Through the analysis of the image data of the reading material, the image text content is obtained, the text content is converted into voice data, and the reading is realized through the speaker. The screen in the speaker with screen can display the preset 3D image with face feature data, and click the preset 3D image with face feature data to realize the interesting point of reading, such as the preset with face feature data The 3D image is the teacher, through the teacher’s point reading, the simulation of real teaching is realized, the learning fun is improved, and the learning efficiency is improved.

The user can also use the preset 3D image with facial feature data displayed in the speaker with a screen, and use the model action method in the above disclosed embodiment to realize the interaction with the preset 3D image with facial feature data to improve the interaction Experience.

The speaker with a screen provided in this embodiment can simulate facial motions through a preset 3D image with facial feature data, improve the effect of facial motion simulation, enhance the authenticity of facial motion simulation, and improve interactive experience.

Example Six

Referring now to FIG. 7, FIG. 7 shows a schematic structural diagram of an electronic device 600 suitable for implementing embodiments of the present disclosure. The electronic devices in the embodiments of the present disclosure may include, but are not limited to, mobile phones, notebook computers, digital broadcast receivers, personal digital assistants (Personal Digital Assistant, PDA), tablet computers (PAD), and portable multimedia players (Portable Media Player). , PMP), mobile terminals such as car navigation terminals, etc., and fixed terminals such as digital televisions (Television, TV), desktop computers, etc. The electronic device shown in FIG. 7 is only an example, and should not bring any limitation to the function and scope of use of the embodiments of the present disclosure.

As shown in FIG. 7, the electronic device 600 may include a processing device (such as a central processing unit, a graphics processor, etc.) 601, which may be based on a program stored in a read-only memory (Read-Only Memory, ROM) 602 or from a storage device 608 is loaded into a random access memory (Random Access Memory, RAM) 603 program to perform various appropriate actions and processing. The RAM 603 also stores various programs and data required for the operation of the electronic device 600. The processing device 601, the ROM 602, and the RAM 603 are connected to each other through a bus 604. An input/output (Input/Output, I/O) interface 605 is also connected to the bus 604.

Generally, the following devices can be connected to the I/O interface 605: including input devices 606 such as touch screen, touch panel, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; including, for example, a liquid crystal display (LCD) Output devices 607 such as speakers, vibrators, etc.; storage devices 608 such as magnetic tapes, hard disks, etc.; and communication devices 609. The communication device 609 may allow the electronic device 600 to perform wireless or wired communication with other devices to exchange data. Although FIG. 7 shows an electronic device 600 having multiple devices, it is not required to implement or have all the illustrated devices. More or fewer devices may be implemented or provided instead.

According to an embodiment of the present disclosure, the process described above with reference to the flowchart may be implemented as a computer software program. For example, the embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program may be downloaded and installed from the network through the communication device 609, or from the storage device 608, or from the ROM 602. When the computer program is executed by the processing device 601, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

The aforementioned computer-readable medium of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination of the above. Examples of computer-readable storage media may include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, RAM, ROM, Erasable Programmable Read-Only Memory (EPROM) Or flash memory), optical fiber, CD-ROM (Compact Disc Read-Only Memory), optical storage device, magnetic storage device, or any suitable combination of the above. In the present disclosure, the computer-readable storage medium may be any tangible medium that contains or stores a program, and the program may be used by or in combination with an instruction execution system, apparatus, or device. In this disclosure, the computer-readable signal medium may include a data signal that is propagated in baseband or as part of a carrier wave, in which computer-readable program code is carried. This propagated data signal can take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. The computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, and the computer-readable signal medium may send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device . The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: electric wires, optical cables, radio frequency (RF), etc., or any suitable combination of the foregoing.

The above-mentioned computer-readable medium may be included in the above-mentioned electronic device; or it may exist alone without being assembled into the electronic device.

The above-mentioned computer-readable medium carries one or more programs, and when the above-mentioned one or more programs are executed by the electronic device, the electronic device: acquires two or more consecutive face images; The face changes in two or more face images are determined to determine the face action feature parameters corresponding to the face changes; the model action instructions are generated according to the face action feature parameters, so as to have the face feature data. It is assumed that the 3D avatar executes the facial action corresponding to the model action instruction according to the model action instruction.

The computer program code for performing the operations of the present disclosure can be written in one or more programming languages or a combination thereof. The above programming languages include object-oriented programming languages such as Java, Smalltalk, C++, as well as conventional Procedural programming language-such as "C" language or similar programming language. The program code can be executed entirely on the user's computer, partly on the user's computer, executed as an independent software package, partly on the user's computer and partly executed on a remote computer, or entirely executed on the remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any kind of network-including Local Area Network (LAN) or Wide Area Network (WAN)-or it can be connected to an external computer ( For example, use an Internet service provider to connect via the Internet).

The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation of the system architecture, functions, and operations of the system, method, and computer program product according to the embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, program segment, or part of code that contains one or more logic functions Executable instructions. It should also be noted that, in some alternative implementations, the functions marked in the block may also occur in a different order from the order marked in the drawings. For example, two blocks shown in succession can actually be executed substantially in parallel, or they can sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and/or flowcharts, and combinations of blocks in the block diagrams and/or flowcharts, can be implemented with dedicated hardware-based systems that perform specified functions or operations Or, it can be realized by a combination of dedicated hardware and computer instructions.

The units described in the embodiments of the present disclosure may be implemented in software or hardware. Among them, the name of the unit in one case does not constitute a limitation on the unit itself. For example, the preset 3D image action execution module can also be described as an "action execution module".

Example 7

The seventh embodiment of the present disclosure also provides a computer-readable storage medium on which a computer program is stored. When the program is executed by the model action device, the model action method as provided in the embodiment of the present disclosure is realized, and the method includes: obtaining continuous Two or more face images; according to the face changes in the two or more face images, determine the face action feature parameters corresponding to the face changes; according to the face action The feature parameter generates a model action instruction, so that the preset 3D avatar with facial feature data executes the face action corresponding to the model action instruction according to the model action instruction.

The computer-readable storage medium provided by the embodiment of the present disclosure is not limited to the implementation of the above method operation when the computer program stored thereon is executed, and can also implement the model action method provided in any embodiment of the present disclosure. Related operations.

Through the above description of the embodiments, the present disclosure can be implemented by software and necessary general-purpose hardware, and of course, it can also be implemented by hardware. Based on this understanding, the technical solution of the present disclosure can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, ROM, RAM, flash memory (FLASH), hard disk, or optical disk. Etc., including multiple instructions to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute the method described in the embodiments of the present disclosure.

In the above embodiment of the model action device, the included units and modules are only divided according to the functional logic, but are not limited to the above division, as long as the corresponding functions can be realized; in addition, the names of the functional units are only for It is easy to distinguish each other and is not used to limit the protection scope of the present disclosure.

Claims

A model action method, including:

Acquire two or more consecutive face images;

Determining the facial motion feature parameters corresponding to the facial changes according to the facial changes in the two or more facial images;

A model action instruction is generated according to the facial action feature parameters, so that a preset three-dimensional 3D image with face feature data executes a face action corresponding to the model action instruction according to the model action instruction.
The method according to claim 1, wherein a model action instruction is generated according to the facial action feature parameters, so that a preset 3D image with face feature data executes a person corresponding to the model action instruction according to the model action instruction Before facial movements, it also includes:

Acquiring at least one face image of a preset angle, and extracting face feature data of the face image of the at least one preset angle;

Constructing a 3D face model corresponding to the face image of the at least one preset angle according to the face feature data;

The 3D face model is applied to a preset 3D image to obtain the preset 3D image with the face feature data.
The method according to claim 1, wherein the facial motion characteristic parameters include at least one of the following parameters: moving speed, moving direction, and moving distance.
The method according to claim 3, wherein the determining the facial motion feature parameters corresponding to the facial changes according to the facial changes in the two or more facial images comprises:

According to the position changes of the preset parts of the face in the two or more face images, the moving speed, the moving direction and the moving distance of the preset parts are determined to determine the facial motion characteristic parameters.
The method according to claim 4, wherein said generating a model action instruction according to said facial action feature parameters, so that a preset 3D image with face feature data is executed according to said model action instruction and said model action instruction The corresponding facial actions include:

A model action instruction is generated according to the movement speed, the movement direction, and the movement distance, so that a preset 3D image with facial feature data executes a face action corresponding to the model action instruction according to the model action instruction.
The method according to claim 1, wherein the preset 3D image with facial feature data is: a cartoon 3D image, a professional 3D image, or a gender 3D image.
A model action device, including:

The face image acquisition module is set to acquire two or more consecutive face images;

The facial motion feature parameter determination module is configured to determine the facial motion feature parameter corresponding to the face change according to the face changes in the two or more face images;

The preset three-dimensional 3D image action execution module is configured to generate model action instructions according to the facial action feature parameters, so that the preset 3D image with face feature data executes the model action instructions corresponding to the model action instructions according to the model action instructions Face action.
The device according to claim 7, further comprising:

The preset 3D image acquisition module is configured to generate model action instructions according to the facial action feature parameters, so that the preset 3D image with face feature data executes the person corresponding to the model action instruction according to the model action instructions Before the facial action, obtain at least one face image with a preset angle, extract the face feature data of the face image at the at least one preset angle; construct the at least one face image with a preset angle according to the face feature data A human face 3D model corresponding to the human face image; applying the human face 3D model to a preset 3D image to obtain the preset 3D image with the facial feature data.
8. The device according to claim 7, wherein the facial motion characteristic parameters include at least one of the following parameters: moving speed, moving direction, and moving distance.
The device according to claim 9, wherein the facial motion feature parameter determination module is configured to: change according to the position change of the preset part of the human face in the two or more facial images, The moving speed, moving direction and moving distance of the human face at the preset position are determined to determine the facial motion characteristic parameters.
The device according to claim 10, wherein the preset 3D image action execution module is configured to: generate model action instructions according to the moving speed, the moving direction and the moving distance, so as to have facial features The preset 3D image of the data executes the facial action corresponding to the model action instruction according to the model action instruction.
7. The device according to claim 7, wherein the preset 3D image with facial feature data is: a cartoon 3D image, a professional 3D image, or a gender 3D image.
A speaker with a screen, comprising a main body, a controller located in the main body, and at least two cameras located on the main body; the distance between the at least two cameras is greater than a distance threshold, and the controller is set as The model action device according to any one of claims 7-12.
An electronic device, including:

One or more processors;

Memory, set to store one or more programs;

When the one or more programs are executed by the one or more processors, the one or more processors implement the model action method according to any one of claims 1-6.
A computer-readable storage medium storing a computer program, which, when executed by a processor, realizes the model action method according to any one of claims 1-6.