CN115330848A

CN115330848A - Facial expression migration method, device and system for three-dimensional facial reconstruction

Info

Publication number: CN115330848A
Application number: CN202210331887.7A
Authority: CN
Inventors: 不公告发明人
Original assignee: Odin Information Technology Co ltd
Current assignee: Odin Information Technology Co ltd
Priority date: 2022-03-31
Filing date: 2022-03-31
Publication date: 2022-11-11

Abstract

The present disclosure provides a facial expression migration method, apparatus and system for three-dimensional facial reconstruction, which includes receiving facial expression animations obtained from at least two angles simultaneously captured, each angle containing M frames of facial expression animations; generating M frames of facial expression 3D models according to the obtained facial expression animation by using an SFM algorithm; calculating M frames of human face expression 3D models into models with the same topology by using a wrapping algorithm; calculating a weight value in the RBF algorithm by using the RBF algorithm based on a target face 3D model in a target face initial state and a facial expression 3D model in a facial expression initial state, wherein the target face initial state and the facial expression in the facial expression initial state are the same; and calculating a target human face 3D model based on the M frames of human face expression 3D models and the weight values by using an RBF algorithm. The embodiment of the disclosure can realize the transfer of expression details to the target face, and can utilize a collection device which has lower cost and is convenient to wear.

Description

Facial expression migration method, device and system for three-dimensional facial reconstruction

Technical Field

The present disclosure relates to three-dimensional face reconstruction technology, and in particular, to a facial expression migration method and apparatus for three-dimensional face reconstruction, and a three-dimensional face reconstruction system.

Background

Industries such as movie/television, games, new media for advertising, virtual reality, etc. require virtual human beings composed of a large amount of realistic three-dimensional data. However, the creation of expressive animations of such three-dimensional data virtual humans is highly manual and time consuming and expensive. Although three-dimensional data of a scanned person can be generated based on a photo taken by an array composed of a plurality of cameras (photo cameras) at multiple angles, because facial expressions of human faces are richer and finer, animation information of the facial expressions can be lost in the shooting mode, and consequently, more manual modeling and animation processing time is needed to make virtual facial expression animation more vivid.

The existing head-mounted face capturing system usually adopts a single small-sized camera to identify and capture some key points of a human face, usually, the capturing only obtains a small number of two-dimensional key points around eyes, a nose and lips of a human, the low-precision data can usually only restore a small amount of information of facial expressions, and a large amount of manual facial expression modeling work is needed in the later period to enable virtual human beings to have relatively natural expression animations.

Moreover, the performer driving the virtual human is inconsistent with the three-dimensional face model of the virtual human prototype, so that the performer expression animation is expected to migrate to the target model, and still has a huge problem. The driving mode of a few sampling points cannot keep all expression details of the performer and transfer the expression details to the target face, and cannot reach the performance at the micro-expression level. The virtual human is just like the real human, and is difficult to reach at present, so that the virtual human plays movies and televisions in a fake and genuine way.

Therefore, a device of a facial expression migration method for three-dimensional facial reconstruction needs to be provided, which can obtain a three-dimensional image with expression information by using two-dimensional images captured by two or more cameras, so as to realize the migration of expression details of a performer to a target face, and thus, the acquisition and reconstruction process of facial expressions of the face has better effect and higher efficiency. In addition, the three-dimensional face reconstruction system designed for realizing the method can realize the acquisition of facial expression information by using portable equipment with lower cost and convenient wearing.

Disclosure of Invention

In order to solve at least one of the above technical problems, according to an aspect of an embodiment of the present disclosure, there is provided a facial expression migration method for three-dimensional facial reconstruction, including receiving facial expression animations obtained by simultaneously photographing from at least two angles, each angle containing M frames of the facial expression animations; using an SFM algorithm to generate M frames of facial expression 3D models according to facial expression animations obtained by shooting from at least two angles simultaneously; calculating the M frames of facial expression 3D models into M frames of facial expression 3D models with the same topology by using a wrapping algorithm; calculating a weight value in the RBF algorithm by using the RBF algorithm based on a target face 3D model in a target face initial state and a facial expression 3D model in a facial expression initial state, wherein the target face initial state and the facial expression in the facial expression initial state are the same; and calculating the M frames of target face 3D models based on the M frames of facial expression 3D models with the same topology and the weight values by using an RBF algorithm.

According to the facial expression migration method for three-dimensional facial reconstruction of the embodiment of the present disclosure, optionally, the method includes simultaneously shooting the performer from at least two angles by at least two cameras to obtain the facial expression animation of the performer, and the shooting range of any one camera and at least one of the other cameras has an overlapping rate of more than 30%.

According to the facial expression migration method for three-dimensional facial reconstruction in the embodiment of the present disclosure, optionally, the same topology indicates that each of the M frames of facial expression 3D models has the same number of vertices, and facial skin regions represented by vertices with the same sequence number in each of the facial expression 3D models are the same.

According to the facial expression migration method for three-dimensional facial reconstruction in the embodiment of the present disclosure, optionally, the target face initial state and the facial expression initial state are non-expression states.

According to the facial expression migration method for three-dimensional facial reconstruction of the embodiment of the present disclosure, optionally, calculating a weight value in an RBF algorithm based on a target face 3D model in a target face initial state and a facial expression 3D model in a facial expression initial state using the RBF algorithm includes setting an RBF interpolation function as:

the function is used as a mapping relation of fitting a facial expression 3D model to a target facial 3D model, wherein N is the sameThe number of vertices of the topological model,

in order to be the basis function(s),

taking the vertex of the target face 3D model in the initial state of the target face as y, taking the vertex of the facial expression 3D model in the initial state of the facial expression as x, and substituting the vertex into the RBF interpolation function to obtain the following equation:

wherein

The N weight values w are obtained by solving the above equation.

According to the facial expression migration method for three-dimensional facial reconstruction of the embodiment of the present disclosure, optionally, the at least two cameras photograph a pattern fixed with respect to the head position of the performer to determine the three-dimensional positions of the respective cameras.

According to another aspect of the embodiments of the present disclosure, there is provided a facial expression migration apparatus for three-dimensional facial reconstruction, comprising a processor and a memory, the memory being configured to store program instructions, and the processor being configured to invoke the program instructions to perform the above method.

According to still another aspect of embodiments of the present disclosure, there is provided a three-dimensional facial reconstruction system, which includes a facial expression capturing apparatus, a synchronization device, and a processing unit, wherein the facial expression capturing apparatus includes at least two cameras, which shoot faces of performers at least two angles, respectively obtaining M frames of facial expression animations; the synchronization device includes a synchronization box that transmits signals for synchronous photographing to the at least two cameras; the processing unit receives facial expression animations obtained by shooting from at least two angles simultaneously, wherein each angle comprises M frames of facial expression animations; using an SFM algorithm to simultaneously shoot the obtained facial expression animation according to at least two angles to generate M frames of facial expression 3D models; calculating the M frames of facial expression 3D models into M frames of facial expression 3D models with the same topology by using a wrapping algorithm; calculating a weight value in the RBF algorithm by using the RBF algorithm based on a target face 3D model in a target face initial state and a facial expression 3D model in a facial expression initial state, wherein the target face initial state is the same as the expression in the facial expression initial state; and calculating the M frames of target face 3D models based on the M frames of facial expression 3D models with the same topology and the weight values by using an RBF algorithm.

According to the three-dimensional face reconstruction system of the embodiment of the present disclosure, optionally, each of the M frames of the facial expression 3D models has the same number of vertices, and the facial skin regions represented by the vertices with the same sequence number in each of the facial expression 3D models are the same.

According to the three-dimensional face reconstruction system of the embodiment of the present disclosure, optionally, the target face initial state and the facial expression initial state are non-expression states.

According to the three-dimensional face reconstruction system of the embodiment of the present disclosure, optionally, calculating a weight value in an RBF algorithm based on a target face 3D model in a target face initial state and a facial expression 3D model in a facial expression initial state using the RBF algorithm includes setting an RBF interpolation function as follows:

the function is used as a mapping relation of a facial expression 3D model fitted to a target facial 3D model, wherein N is the number of vertexes of the same topological model,

is a function of the basis function(s),

wherein

The N weight values w are obtained by solving the above equation.

According to the three-dimensional facial reconstruction system of the embodiment of the present disclosure, optionally, the facial expression capturing apparatus further comprises a head-mounted frame set including an upper frame to be worn above the head, two connecting members respectively connected to two ends of the upper frame, a front frame connected to at least one of the connecting members and extending to the front of the face, the at least two cameras being disposed on the front frame; and a lighting part disposed on the front frame; wherein a predetermined pattern is provided on a surface of the upper frame facing the at least two cameras, the predetermined pattern being fixed in position relative to the head of the performer.

According to the three-dimensional face reconstruction system of the embodiment of the present disclosure, optionally, the head-mounted frame set further includes a rear frame including an adjustment rotation shaft and two links symmetrically disposed, wherein one end of each link is inserted into the adjustment rotation shaft, and the other end of each link is rotatably connected to one of the two links, and the adjustment rotation shaft is rotatable to adjust a depth into which each link is inserted.

According to the three-dimensional facial reconstruction system of the embodiment of the present disclosure, optionally, the head-mounted frame set further includes a mandible fixing band which is attached to the mandible of the head when being worn.

According to the three-dimensional facial reconstruction system of the embodiment of the present disclosure, optionally, both ends of the front frame are respectively fixed to the two connecting pieces, and the front frame includes one or more struts.

According to the three-dimensional face reconstruction system of the embodiment of the present disclosure, optionally, both ends of the front frame are respectively fixed to the connecting rods of the rear frame, and the front frame includes one or more struts.

According to the three-dimensional face reconstruction system of the embodiment of the present disclosure, optionally, the front frame includes two or more U-shaped struts juxtaposed and attached together at both sides of the face and laterally spaced apart in front of the face.

According to the three-dimensional face reconstruction system of the embodiment of the present disclosure, optionally, the lighting component disposed on the front frame includes two or more LED lamps and is disposed in at least one supporting rod, wherein the supporting rod is made of a transparent material.

According to the three-dimensional face reconstruction system of the embodiment of the present disclosure, optionally, the two connecting members are adapted to the shape of the auricle to be worn behind the ear, the two ends of the upper frame are rotatably connected to the two connecting members, and the pattern on the upper frame is a single-color dot pattern.

According to the three-dimensional face reconstruction system of the embodiment of the present disclosure, optionally, the synchronizing box is configured to be worn on the performer.

The facial expression migration method for three-dimensional facial reconstruction provided by the embodiment of the disclosure can adopt a plurality of high-speed global shutter cameras, and solves the problem of expression detail acquisition. The method of the embodiment of the disclosure utilizes the RBF algorithm to transfer the expression animation data to the target face, so that the target face can obtain the micro-expression level 3D expression animation data. Different from the traditional face capture method which uses sampling point mapping, the method obtains more complete micro-expression animation details, and transfers the details to the target face 3D model, so that the virtual human can make a false-to-true micro-expression animation.

In addition, the three-dimensional face reconstruction system provided by the embodiment of the disclosure can adopt the synchronization box to realize synchronization of the shooting process of the camera on one hand, and can utilize the portable facial expression capture device to realize the collection of the expression details on the other hand. One or more parts in the head-mounted frame group in the system can be stably fixed on the head, so that clear and fine facial expression images can be obtained conveniently. In addition, a plurality of components in the head-mounted frame set can be set to be in an adjustable mode, so that the facial expression capturing device can be conveniently matched with different acquisition objects and acquisition scenes. Therefore, the camera matrix can keep a relatively stable relation with the face, and a performer can freely move the head to perform expression performance, so that vivid and natural expressions are captured.

In addition, the preset patterns are arranged to provide the basis of alignment for the patterns shot by the camera matrix formed by at least two cameras, so that expression animation data can be more stable and natural, the processing efficiency and the effect are improved, and the fidelity is also obviously improved.

Moreover, the three-dimensional facial reconstruction system comprising the facial expression capturing device can set the image capturing part and the control part thereof as a portable device, and automatically complete the three-dimensional facial reconstruction processing to the maximum extent, so that the three-dimensional facial reconstruction system is convenient to use and has lower cost.

It is not necessary for any device embodying the present disclosure to achieve all of the above-described advantages at the same time. Additional features and advantages of the disclosure will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by the practice of the disclosure. The objects and advantages of the embodiments of the disclosure may be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.

Drawings

To more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings of the embodiments will be briefly described below, and it should be apparent that the drawings in the following description are only related to some embodiments of the present disclosure, and do not limit the present disclosure.

Fig. 1 is a flow diagram of a facial expression migration method for three-dimensional facial reconstruction according to one embodiment of the present disclosure;

FIG. 2 is a schematic diagram of multiple cameras capturing multi-angle video and generating a 3D model according to one embodiment of the present disclosure;

FIG. 3 is a schematic diagram of processing an emoji animation per-frame 3D model into the same topological mesh using a wrapping algorithm, according to one embodiment of the present disclosure;

FIG. 4 is a schematic diagram of generating a target face 3D model based on M frames of facially expressed 3D models of the same topology using RBF algorithm according to one embodiment of the present disclosure;

FIG. 5 is a schematic diagram of a three-dimensional face reconstruction system according to one embodiment of the present disclosure;

FIG. 6 is a schematic side view of a portable facial expression capture device when worn according to one embodiment of the present disclosure; and

FIG. 7 is an exploded schematic view of a portable facial expression capture device according to one embodiment of the present disclosure.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present disclosure more apparent, the technical solutions of the embodiments of the present disclosure will be described clearly and completely with reference to the drawings of the embodiments of the present disclosure. It is to be understood that the described embodiments are only a few embodiments of the present disclosure, and not all embodiments. Various embodiments may be combined with each other to form other embodiments not shown in the following description. All other embodiments, which can be derived by a person skilled in the art from the described embodiments of the disclosure without any inventive step, are within the scope of protection of the disclosure.

Unless defined otherwise, technical or scientific terms used herein shall have the ordinary meaning as understood by one of ordinary skill in the art to which this disclosure belongs. The use of "first," "second," and similar terms in the description and claims of the present disclosure are not intended to indicate any order, quantity, or importance, but rather are used to distinguish one element from another. Also, the use of the terms "a" or "an" and the like do not necessarily denote a limitation of quantity. The word "comprising" or "comprises", and the like, means that the element or item preceding the word comprises the element or item listed after the word and its equivalent, but does not exclude other elements or items. The terms "connected" or "coupled" and the like are not restricted to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "upper", "lower", "left", "right", and the like are used merely to indicate relative positional relationships, and when the absolute position of the object being described is changed, the relative positional relationships may also be changed accordingly.

Fig. 1 is a flow chart of a facial expression migration method for three-dimensional facial reconstruction according to an embodiment of the present disclosure, which mainly includes steps 101-107. These steps are described in detail below in conjunction with fig. 2-5.

As shown in fig. 1, M frames of images of a certain duration, which are captured by at least two cameras from multiple angles, are received in step 101. Cameras used for photographing are disposed at different positions with respect to an actor (i.e., a scanned subject) so as to photograph the photographed subject from different angles. In order to ensure that the expression animation of the shot object can be collected without motion blur, the camera can adopt a high-speed camera with a global shutter. Here, the shooting rate is not lower than 30 frames per second, and for example, 55 frames per second may be used for shooting. The shooting range of the camera array covers the whole area of the human face, and any one camera and at least one of the rest cameras have a certain overlapping rate in the shooting range, such as more than 30%. For example, the shooting ranges of adjacent cameras in the camera array have an overlap ratio of 30% or more.

Fig. 5 is a schematic diagram of a three-dimensional face reconstruction system according to one embodiment of the present disclosure. The system includes a facial expression capture apparatus 50, a processing unit 80, and a synchronization device 90. In this embodiment, the facial expression capture device 50 includes a camera array of 6 cameras, each of which captures a face simultaneously from different locations. In order to ensure that the cameras shoot synchronously, the synchronization device 90 of the three-dimensional face reconstruction system sends signals for synchronous shooting to the 6 cameras. The processing unit 80 shown in fig. 5 receives the photographed M-frame animation from each camera (which may be via the synchronization device 90 or by other means, wireless or wired), whereby the same number of multi-angle images as the number of cameras can be obtained per frame. The processing unit 80 will process the received image to effect the transfer of the expression.

Referring to fig. 1, in step 102, a 3D model is generated for each frame, thereby producing an M-frame 3D model. Specifically, an SFM algorithm may be employed to generate M3D models of the facial expressions of the performer based on M frames of animation received from each camera. These model data record the facial expressions of each frame of the performer.

To obtain the 3D model, the relative position of each camera needs to be located first. As shown in fig. 5, the facial expression capture device 50 includes a predetermined pattern 520 that is fixed relative to the performer's head. The preset pattern provides an alignment basis for each frame of image shot by a camera matrix consisting of at least two cameras, so that a 3D model can be generated conveniently.

Fig. 2 shows a schematic diagram of a multi-angle video taken with 6 cameras and generating a 3D model using the three-dimensional face reconstruction system shown in fig. 5. As shown in fig. 2, 6 cameras (201, 203, 205, 207, 209, 211) respectively photograph the performer from 6 different angles, and each obtain M-frame images. Each frame has the same number of multi-angle images as cameras. After the SFM algorithm, a 3D model is generated for each frame, thereby obtaining M3D models 220 that record facial expression models for each frame.

Referring back to fig. 1, in step 103, the processing unit 80 calculates each frame model as 3D model data of the same topology based on the M3D models obtained in step 102. The meaning of the same topology may be that each of the M frames of 3D models has the same number of vertices, and the vertices with the same sequence number in each model represent the same face skin region. Referring specifically to fig. 3, the 3D model 310 generated by the SFM algorithm is processed into a 3D model 320 of the same topology, for example, triangular face meshes of the same topology may be generated using a wrapping algorithm.

In step 104, the processing unit 80 generates a model having the same topology as the model in step 103 as a 0 th frame model for the facial expression initial state. This frame may record the expressionless status of the performer. In step 105, a target human face 3D model of the virtual human is obtained by authoring with a modeling tool or using a scanning device, etc., which is topologically the same as the model in step 104. The expression state of the target face 3D model in the initial state is the same as the expression state of the facial expression initial state, for example, the target face 3D model may be a non-expression state. Specific examples can be seen in diagram 410 of fig. 4, which shows a 0 th frame model of facial expression and a corresponding target face model of a avatar. The order of

steps

103, 104, 105 may be interchanged. The target face 3D model may be preset before acquiring the real facial expression.

In step 106, the processing unit 80 calculates a weight value in the RBF algorithm according to the 0 th frame model and the model in the initial state of the target face based on the RBF algorithm. The method of calculating the weight value is specifically described below.

First, the mapping relationship from the model space of the performer's face to the model space of the target human face is fitted using the following RBF interpolation function:

wherein N is the number of vertexes of the same topological model,

is a basis function. The basis function may be a gaussian basis function or the like.

Substituting the vertex of the model in the initial state of the target face obtained in step 105 as y and the vertex of the model in the 0 th frame of the performer obtained in step 104 as x into the RBF interpolation function to obtain the above equation (2).

By solving equation (2) above, N weight values w can be obtained ₁ ……w _N 。

In step 107, the processing unit 80 substitutes each vertex of the model for each frame in the M frames obtained in step 103 into the above equation as x, x _i The vertices of the model of the 0 th frame, and N is the number of model vertices. Therefore, each frame of target face 3D model can be obtained according to all vertex data of each frame of face expression 3D model, all M frames of models are sequentially brought in, and M target face 3D models can be obtained. As shown in FIG. 4, the progression of model 420 through model 430This migration process is shown.

The method is different from a method of using sampling point mapping for face capture, obtains more complete micro-expression animation details, and migrates the details to a target face 3D model, so that a virtual human can make a micro-expression animation which is fake and real.

In order to obtain data suitable for processing by the method, at least two cameras are adopted to shoot M frames of images of the performer at different angles for a certain time, and accurate positioning of the two cameras is required. To obtain the natural expression of the performer, the acquisition of the performer's expression may be accomplished using a wearable or wearable portable device.

As described above, fig. 5 shows a schematic diagram of a three-dimensional facial reconstruction system according to one embodiment of the present disclosure, wherein the system includes a facial expression capture device 50. Fig. 6 and 7 show a schematic side view of the facial expression capture device 50 when worn and an exploded schematic view of the device. Other configurations of facial expression capture devices, portable or non-portable, may also be used.

As shown in fig. 5-7, the portable facial expression capture device 50 includes a head-mounted frame set 500, a camera 600, and an illumination component 700. The head-mount frame set 500 is worn on the head so that the position of the head with the camera 600 and the lighting part 700 can be kept fixed and adjusted as needed. This configuration ensures a relatively stable relationship of the camera 600 with the face of the performer even in the case where the performer is free to move his head.

The control device 90 includes at least a synchronization box 91 to transmit a trigger signal for synchronized shooting to the camera. The synchronization box 91 is connected to each camera in a wired or wireless manner, and is used to change the pulses of fixed frame rate per second generated by the controller 93 into a trigger signal for driving the cameras to capture each frame, so that all the cameras capture each frame synchronously. The controller 93 may control the operation of the sync box 91, and may receive the moving picture data photographed by the camera, store it as an image file corresponding to each frame, and then forward it to the processing unit 80. The functions of the controller 93 may also be implemented in combination with the processing unit 80. The control device 90 may be provided as a portable device to be worn on the performer. The control device 90 and the processing unit 80 may be connected by wire or wirelessly. The processing unit 80 may also be directly connected to the facial expression capture device 50 via a wired or wireless connection.

In the process, due to the introduction of the preset patterns, each camera can accurately reconstruct the three-dimensional position of the camera, so that data obtained by scanning can be aligned to a uniform three-dimensional space, and the image processing efficiency and effect are improved.

The controller 93 may be implemented using a general-purpose device. The control device 90 may further include a display section, a memory, and the like as necessary.

The processing unit 80 may include at least a processor and a memory (e.g., a non-volatile memory), such as may be implemented by a general purpose computer.

The camera 600 includes two or more cameras. A camera matrix consisting of 6 cameras is shown. The number of cameras can be adjusted according to the requirements of different three-dimensional reconstruction accuracy. To be suitable for capturing facial expressions, as described above, the camera 600 may employ a small camera with a frame rate of not less than 30 frames per second, and may further be provided with a global shutter and a high resolution to ensure that no picture blurring problems occur during shooting due to too fast a motion speed of an expression of the performer. To synchronize each frame shot by the multiple cameras, a synchronization box in the control device 90 may be used to send a trigger signal to each camera that synchronizes each frame shot. The data of the camera 600 may be transmitted to the control device 90 or the processing unit 80 by wire or wirelessly.

The lighting unit 700 may include one or more LED lamps for supplementing light to the face. LED lamp sets symmetrical relative to the face can be arranged, so that the face illumination is more uniform. The lighting assembly 700 may also be implemented with other suitable light sources.

Headgear set 500 may include an upper frame 510, a rear frame 530, a front frame 540, a rear brain fixation strap 550, a mandible fixation strap 560, and a connector 570. In order to maintain the head position stable with respect to the camera, the head mount set 500 needs to have at least the upper frame 510, the front frame 540, and the connection member 570.

The upper frame 510 is worn over the head with both ends connected to the first and

second connectors

571 and 573, respectively, of the connectors 570. The upper frame 510 is provided with a preset pattern 520 on a surface facing the camera. The predetermined pattern 520 may be composed of a randomly distributed single color dot pattern, which allows each camera to accurately reconstruct its own three-dimensional position. The preset pattern is arranged on the upper frame, so that each frame of three-dimensional model data of the expression animation can be aligned to the same space coordinate, and the expression animation data are more stable and natural. The predetermined pattern may also be provided on a separate member, which is attached to the surface of the upper frame 510.

The upper frame 510 may also include a positioning block 515 to assist in positioning the upper frame, which may be configured as a stop to conform to the shape of the crown of the head to further prevent movement of the upper frame at the crown portion.

The connection member 570 includes first and

second connection members

571 and 573 to be worn on the left and right sides, and may be provided in a shape suitable for the auricle, for example, a substantially semicircular ring shape, to be positioned behind the auricle when worn. The ends of the upper frame 510 are connected to the first and

second connectors

571 and 573 by a connection mechanism. The upper frame 510 and the first link 571 may be connected in a manner pivotable about the first rotation axis 513, and the upper frame 510 and the second link 573 may be connected in the same or similar manner, so that the upper frame can adjust its position at the top of the head within a certain range. The connection means can be locked so that the upper frame remains stable relative to the head after adjustment into position.

The rear frame 530 includes an adjustment rotating shaft 539 and symmetrically disposed two-

side links

535 and 537. The

connection rods

535 and 537 may take an L-shaped structure, and one ends of the two connection rods are inserted into both sides of the adjustment rotation shaft 539, respectively, and the other ends are connected to the first and

second connection members

571 and 573, respectively. The adjustment rotating shaft 539 can be rotated to adjust the depth of insertion of each link therein to adjust the distance between the links, whereby the degree of clamping of the headgear frame set can be adjusted.

The front frame 540 may be secured to the links of the rear frame 530 as shown, for example by inserting its ends into the links or using other connections, whereby the front frame may be connected to the connectors via the links of the rear frame. To adjust the position of the front frame 540 with respect to the face, the link 535 may be pivotally connected to the first connector 571 by the second rotation shaft 533, and the link 537 and the second connector 573 may be connected in the same or similar manner. The connection of the rotating shaft 533 can be locked when adjusted to a desired position. The front frame 540 is shown as being generally U-shaped, the front frame 540 may also take the shape of, for example, an L and be secured at one end to one of the links. Further, the front frame 540 may be directly connected to one of the connectors (e.g., may take a substantially L-shape) or directly connected to both side connectors (e.g., may take a substantially U-shape) in a fixed or rotatable manner. The front frame 540 may also be provided as a telescopic structure to further adjust the distance and position to the face. By one or more of the above adjustment means, the camera array can be adjusted to a position that can cover the entire face.

Fig. 5-7 show a generally U-shaped front frame 540 comprising two

struts

541 and 543 which fit together at lateral locations, spaced up and down in front of the face.

The number of poles may be one or more, which may be determined according to the location and number of cameras required to be installed. For example, if two cameras are provided, the number of struts may be 1. In the case of multiple struts, the struts may be longitudinally spaced apart. A camera matrix of 6 cameras is shown, arranged on two

symmetrical struts

541 and 543 respectively. The camera may be arranged to be able to be mounted, dismounted or positionally adjusted as required, or the camera may be arranged to be fixed to the bar. The lighting member 700 may be provided on the rod to supplement light to the face when photographing.

The support rod can be made of light rigid materials so as to reduce the weight of the support rod. In addition, the supporting rod can also be made of transparent materials and has a hollow structure. The illumination member 700 may be disposed in the hollow structure of the pole as shown in fig. 5 and 7, making the appearance more concise and aesthetically pleasing. The lighting component 700 may include one or more LED lights. The LED lamps can be symmetrically distributed in the support rod, so that the light supplement of the face is more uniform. The support bar adopts a hollow structure and can also be used for hiding and protecting the wiring of the lighting assembly and/or the camera.

The front frame 540 may also take other shapes or configurations as long as it is convenient to dispose the camera and the illumination components thereon.

The hindbrain fixation band 550 may be made of elastic material and may have a length adjustable structure as required. Both ends of the hindbrain fixation band 550 are fixed to the left and right first and

second connectors

571 and 573, respectively. To further fix the position of the head-mounted frame set 500, a chin securing strap 560 may be further provided, which may be made of an elastic material and may also be of an adjustable length structure as desired. The chin fixing strap 560 is fixed at both ends thereof to the left and right first and

second connectors

571 and 573, respectively.

The facial expression capture device 50 may be connected to an external power source for power or have a rechargeable battery (not shown) disposed thereon. A plurality of components in the head-mounted frame group of the facial expression capturing device are set to be in an adjustable and/or detachable mode, so that the facial expression capturing device can be conveniently matched with different acquisition objects and acquisition scenes.

The above description is intended to be exemplary of the present disclosure, and not to limit the scope of the present disclosure, which is defined by the claims appended hereto.

Claims

1. A facial expression migration method for three-dimensional facial reconstruction includes

Receiving facial expression animations obtained by shooting from at least two angles simultaneously, wherein each angle comprises M frames of facial expression animations;

using an SFM algorithm to simultaneously shoot the obtained facial expression animation according to the at least two angles to generate M frames of facial expression 3D models;

calculating the M frames of facial expression 3D models into M frames of facial expression 3D models with the same topology by using a wrapping algorithm;

calculating a weight value in the RBF algorithm by using the RBF algorithm based on a target face 3D model in a target face initial state and a facial expression 3D model in a facial expression initial state, wherein the target face initial state and the facial expression in the facial expression initial state are the same;

and calculating a target human face 3D model based on the M frames of human face expression 3D models with the same topology and the weight values by using an RBF algorithm.

2. The facial expression migration method for three-dimensional facial reconstruction as claimed in claim 1, comprising simultaneously photographing the performer from at least two angles by at least two cameras to obtain the facial expression animation of the performer, any one of the cameras having an overlap ratio of 30% or more with a photographing range of at least one of the remaining cameras.

3. The facial expression migration method for three-dimensional facial reconstruction according to claim 1, wherein each of the M frames of the 3D models of facial expression has the same number of vertices, and the facial skin regions represented by the vertices with the same sequence number in each 3D model of facial expression are the same.

4. The facial expression migration method for three-dimensional facial reconstruction as claimed in claim 3, wherein the target face initial state and the facial expression initial state are non-expression states.

5. The facial expression migration method for three-dimensional facial reconstruction as claimed in claim 3 or 4, wherein the calculating of the weight value in the RBF algorithm based on the target face 3D model in the target face initial state and the facial expression 3D model in the facial expression initial state using the RBF algorithm comprises setting the RBF interpolation function as:

in order to be the basis function(s),

wherein

And obtaining N weight values w through solving.

6. The facial expression migration method for three-dimensional facial reconstruction as claimed in claim 2, wherein said at least two cameras photograph a pattern fixed with respect to the position of the performer's head to determine the respective three-dimensional positions.

7. A facial expression migration apparatus for three-dimensional facial reconstruction, comprising a processor and a memory, the memory for storing program instructions, the processor for invoking the program instructions to perform a method comprising any of claims 1 and 3-5.

8. A three-dimensional face reconstruction system is characterized by comprising a facial expression capturing device, a synchronization device and a processing unit, wherein

The facial expression capturing device comprises at least two cameras, and the cameras shoot faces of performers at least two angles to respectively obtain M frames of facial expression animations;

the synchronization apparatus includes a synchronization box which transmits signals for synchronized photographing to the at least two cameras;

the processing unit receives facial expression animations obtained by shooting from at least two angles simultaneously, wherein each angle comprises M frames of facial expression animations; using an SFM algorithm to simultaneously shoot the obtained facial expression animation according to the at least two angles to generate M frames of facial expression 3D models; calculating the M frames of facial expression 3D models into M frames of facial expression 3D models with the same topology by using a wrapping algorithm; calculating a weight value in the RBF algorithm by using the RBF algorithm based on a target face 3D model in a target face initial state and a facial expression 3D model in a facial expression initial state, wherein the target face initial state is the same as the expression in the facial expression initial state; and calculating a target face 3D model based on the M frames of face expression 3D models with the same topology and the weight value by using an RBF algorithm.

9. A three-dimensional face reconstruction system as claimed in claim 8, wherein any one of the cameras has an overlapping rate of 30% or more with a photographing range of at least one of the remaining cameras.

10. The three-dimensional facial reconstruction system of claim 8 wherein said identical topology represents that each of the M frames of facially expressed 3D models has the same number of vertices and that the same number of vertices in each of the facially expressed 3D models represent the same area of facial skin.

11. The three-dimensional facial reconstruction system according to claim 10, wherein said target face initial state and said facial expression initial state are non-expression states.

12. The three-dimensional facial reconstruction system according to claim 10 or 11, wherein calculating the weight value in the RBF algorithm based on the target face 3D model in the target face initial state and the facial expression 3D model in the facial expression initial state using the RBF algorithm comprises setting an RBF interpolation function to:

in order to be the basis function(s),

wherein

The N weight values w are obtained by solving this equation.

13. The three-dimensional facial reconstruction system of claim 8 wherein said facial expression capture means further comprises:

the head-mounted frame group comprises an upper frame which is worn above the head, two connecting pieces which are respectively connected with two tail ends of the upper frame, and a front frame which is connected with at least one of the connecting pieces and extends to the front of the face, wherein the at least two cameras are arranged on the front frame; and

a lighting part disposed on the front frame;

wherein the upper frame has a predetermined pattern on a surface thereof facing the at least two cameras, the predetermined pattern being fixed in position relative to the head of the performer.

14. The three-dimensional face reconstruction system of claim 13 wherein said set of head frames further comprises a rear frame including an adjustment rotation shaft and two links symmetrically disposed, wherein one end of each link is inserted into the adjustment rotation shaft, and the other end of each link is rotatably connected to one of the two said links, and said adjustment rotation shaft is rotatable to adjust a depth of insertion of each link thereinto.