TWM508085U - System for generating three-dimensional facial image and device thereof - Google Patents

Abstract

A system for generating a three-dimensional facial image is disclosed. The system includes a server and at least a communication device. The communication device is communicated to the server, and stores a doll data from the server in advance. The communication device receives a facial feature data and a facial texture data transmitted from the server. The model data is modified by the communication device in accordance with facial feature data and the facial texture data. The three-dimensional facial image is displayed on the communication device in accordance with the facial feature data and the modified model data. The present invention discloses a device for generating a three-dimensional facial image as well.

Description

Three-dimensional avatar generating system and device thereof

This creation is about a three-dimensional avatar generation system and its apparatus.

In today's life, as the threshold for the establishment of communication devices continues to decline, the widespread use of mobile communication devices, the use of the network world and virtual situations is more and more convenient and easy, and the proportion of modern people's time is getting higher and higher.

Because of the time and emotion invested, users value the importance of operating the "self" in the online or virtual world. In the past, the use of words or numbers as a representation of identity has become apparently insufficient. Even though various communication media or social networking sites have been designed so that users can use photos or photo stickers to identify themselves or highlight their personal style, they are still in the flat stage. It's impossible to let the user create a virtual feeling that they can jump on the screen.

In order to solve this problem, a technology of virtual dolls or dolls (Avatar in English) has been developed, mainly to create a three-dimensional avatar similar to oneself on an electronic device, and then develop a human figure as a A representative of the Internet or the virtual world. However, the application of this technology is only limited to the establishment of a large number of facial features, hairstyles, face and body modules in the database, so that users can select modules to combine according to their own image. Even the combined modules can be manually fine-tuned But the difference in shape between people is so diverse, using a limited number of modules instead of virtual dolls that are really difficult to produce a truly highly similar look.

In view of this, the author thinks of a three-dimensional avatar generation system and its communication device, through the combination of the body and the user's face related data, to generate a three-dimensional avatar with high similarity, and because the body is pre-stored in use In the device, the face related data is received and does not need to be processed by the device itself, so it helps to reduce the processing time and the hardware threshold of the device. Once you have a very similar 3D avatar, users can easily create highly self-representative or recognizable virtual dolls or dolls to increase the use of the online world or virtual context.

The purpose of the present invention is to provide a three-dimensional avatar generating system and a communication device thereof, which combines the physical body and the face data of the user to generate a three-dimensional avatar with high similarity, and because the primal body is a device pre-stored in the user. Inside, the facial related data is received and does not need to be processed by the device itself, so it helps to reduce the processing time and the hardware threshold of the device. Once you have a very similar 3D avatar, users can easily create highly self-representative or recognizable virtual dolls or dolls to increase the use of the online world or virtual context.

In the present creation, the so-called avatar does not necessarily include the entire human head in biology or physiology, but at least includes the face. In other words, the three-dimensional avatar produced by this creation is mainly to produce a three-dimensional face, and is not limited by the position where the hair or the hindbrain to be included does not cause significant differences due to different people.

To achieve the above object, a three-dimensional avatar generating system according to the present invention includes a server and at least one communication device. The communication device is connected to the server and stored in advance Save one of the physical data from the server. The server transmits a facial feature data and a facial model data to the communication device. The communication device adjusts the physical body data according to the facial feature data and the facial model data, and the communication device generates a three-dimensional avatar according to the facial model data and the adjusted physical body data.

In order to achieve the above object, a three-dimensional avatar generating device according to the present invention is provided. The three-dimensional avatar generating device comprises a transmission unit, a storage unit and a processing unit. The storage unit stores a piece of physical data in advance. The processing unit is electrically connected to the transmission unit and the storage unit, respectively. The processing unit adjusts the physical body data according to the facial feature data and the facial model data, and generates a three-dimensional avatar according to the facial model data and the adjusted physical body data.

In one embodiment, the primal data is from a server.

In an embodiment, the facial feature data or the facial model data is obtained by a server according to a planar avatar, and the planar avatar corresponds to the three-dimensional avatar.

In an embodiment, the facial feature data includes a plurality of facial feature points, the physical material includes at least one feature region, the feature region includes a plurality of feature region feature points, a plurality of facial feature points, and a plurality of feature region feature points Correspondingly, the processing unit adjusts the spatial coordinates of the feature points of the plurality of feature regions according to the plurality of facial feature points.

In an embodiment, the facial model data includes a plurality of facial pair sites, the body data includes a plurality of body pair sites, and the plurality of face pairs and the plurality of body pairs correspond to each other. The processing unit combines the facial model data with the physical data.

In one embodiment, the processing unit changes the spatial coordinates of a portion of the physical data based on the facial model data.

1‧‧‧3D avatar generation system

2‧‧‧Communication device

3‧‧‧Server

21, 31‧‧‧ transmission unit

22, 32‧‧‧ storage unit

23, 33‧ ‧ processing unit

24‧‧‧Display unit

4‧‧‧Characteristic Zone

41‧‧‧Characteristic feature points

5‧‧‧flat avatar

51‧‧‧Face feature points

6‧‧‧Face model data

61‧‧‧ Face alignment

71‧‧‧ Physique alignment

FIG. 1 is a schematic diagram of a system architecture of an embodiment of a three-dimensional avatar generating system.

FIG. 2 is a schematic diagram of the communication device shown in the creative embodiment when displaying the physical material.

3 is a partially enlarged schematic view of the element body shown in FIG. 2 and showing the feature points.

FIG. 4 is a schematic diagram of the result of capturing a facial feature point from a planar avatar in the creative embodiment.

FIG. 5 is a schematic diagram of facial model data in the present embodiment.

FIG. 6 is a schematic diagram of the adjustment of the physical data according to the present embodiment.

FIG. 7 is a schematic diagram of combining a face model with a body alignment according to the present embodiment.

Hereinafter, a three-dimensional avatar generating system and a communication device thereof according to a preferred embodiment of the present invention will be described with reference to related drawings.

1 is a schematic diagram of a system architecture of an embodiment of a three-dimensional avatar generating system. As shown in FIG. 1 , the three-dimensional avatar generating system 1 of the present embodiment includes at least one communication device 2 and a server 3, preferably at the same time. A plurality of communication devices 2 are included for simultaneous operation by a plurality of users.

The communication device 2 can be a smart phone, a tablet computer, a mobile digital assistant, a networked video camera, a wearable device, a desktop computer, a notebook computer, or any device with networking capabilities. In the embodiment, the communication device 2 is described by taking a smart phone as an example, and communicating with the server 3 through the Internet through wireless communication. However, in other embodiments, the communication device 2 can also be at a fixed location. Operating laptop or desktop computer.

The server 3 includes a transmission unit 31, a storage unit 32, and one or more processing units 33. The storage unit 32 and the transmission unit 31 are respectively in communication with the processing unit 33. In the following embodiments, the server 3 performs operations and processing through the processing unit 33, performs data transmission through the transmission unit 31, and stores related materials through the storage unit 32.

The communication device 2 includes a transmission unit 21, a storage unit 22, a processing unit 23, and a display unit 24. The transmission unit 21, the storage unit 22, and the display unit 24 are electrically connected to the processing unit 23, respectively.

The user can first receive the application (APP) from the server 3 by using the transmission unit 21 of the communication device 2, and store the application to the storage unit 22. Since the application includes a doll data, the communication device 2 also downloads the physical data while downloading the application. In other words, the physical data has been previously stored in the storage unit 22 of the communication device 2 before the application is executed. The prime system refers to a model having a partial human body frame or contour, such as a head model, or a whole body model of the present embodiment, and FIG. 2 is a schematic diagram of the communication device shown in the creative embodiment when displaying the physical material.

In this embodiment, after the physical data is turned on by the communication device 2, a three-dimensional humanoid image can be displayed on the display unit 24, and the three-dimensional image includes at least one face. As shown in Figure 2, the physical data includes the complete head, trunk and limbs, the front of the head is the face, and the face has five features such as eyebrows, eyes, ears and nose. The establishment of the physical data can be obtained by the server 3 by downloading the human body data including the facial data through the three-dimensional modeling.

Fig. 3 is a partially enlarged schematic view showing the element body shown in Fig. 2. Referring to FIG. 3, when the physical material of the embodiment is established, the eyebrows, the facial features and the face are defined as feature areas 4, and each of the feature areas 4 has a plurality of feature area feature points 41. Taking the eye as an example, the feature area feature points 41 are arranged according to the contours around the eyes, in other words, the feature area feature points 41 in the eye feature area 4 are arranged to outline the outline of the eyes. The spatial coordinate position of each feature area feature point 41 is recorded in the elemental data, respectively. The method for generating the spatial coordinate position may be that the center point of the face is used as a reference coordinate, and the relative space coordinates of the feature points 41 of each feature area are calculated. In addition, each feature area feature point 41 has a respective registration number. In this embodiment, there are a total of eightty-seven feature area feature points 41 distributed in the feature area 4 such as, but not limited to, eyebrows, eyes, nose, mouth, ears, and the like. Within, so the number is numbered sequentially from number one to eighty-seven, as a means of identification between points. It should be noted that in order to avoid the drawing being too complicated and difficult to understand, the eighty-seven feature area feature points 41 are not fully marked in FIG.

In the present embodiment, as shown in FIG. 2 and FIG. 3, displaying the physical data through the communication device 2 is not a necessary step for generating a three-dimensional avatar, that is, the physical data is not necessarily displayed after being stored, and may be simply stored in the storage unit 22. For later use.

When the user wishes to create a three-dimensional avatar, the application in the communication device 2 can be activated and an image can be uploaded to the server 3. Server 3 will analyze the image after receiving it. In this embodiment, the user can use the communication device 2 to take a picture of his own plane, that is, a photo with a face image, and upload it to the server 3 for analysis. Of course, in other embodiments, photos or images that have been stored in the communication device 2 or anywhere else may be used instead.

After the planar avatar is transmitted to the server 3, the processing unit 33 of the server 3 can recognize the facial features in the planar avatar according to an algorithm or a software program to form a facial feature. data. Specifically, the processing unit 33 can identify the plane avatar according to an algorithm or a software program related to image recognition, so as to distinguish between specific regions having facial features, such as, but not limited to, eyebrows, facial features, and face types, and then multiple numbers. The points are arranged to outline the areas. Then, the server 3 captures the points as facial feature points, and then combines the facial feature points or forms the facial feature data together with other contents to obtain the features of the face. FIG. 4 is a schematic diagram of the result of extracting facial feature points from the planar avatar 5 in the embodiment. Referring to FIG. 4, in this embodiment, the plane avatar 5 is analyzed by an Active Appearance Model (AAM), and a total of eighty-seven facial feature points 51 are obtained. The eighty-seven facial feature points 51 also have a registration number and correspond to the feature area feature points 41 of the physical data for future adjustment of the facial face features. In order to avoid the drawing being too complicated and difficult to understand, the eighty-seven facial feature points 51 are also not fully marked in FIG.

Of course, in order to improve the effectiveness of the active appearance model algorithm, one or more sets of reference images may be trained before use. In addition, in order to further improve the active appearance model algorithm, in the process of capturing the facial feature points 51, it is also possible to simultaneously match the model prediction and the processing of the skin color range differentiation in the YCbCr color space.

At the same time, the processing unit 33 of the server 3 also recognizes the planar avatar 5 shown in FIG. 4 to generate a facial model data. A large number of face models can be stored in the storage unit 32 of the server 3, but there is a slight difference between the models. The processing unit 33 of the server 3 can divide the set of facial feature points 51 into a coordinate system according to the geometric center position of the set of facial feature points 51 as described above, and according to the central position and each face. The distance between the feature points 51 and the angle are similarly calculated, so that the similarity is higher in the face model database. Please refer to Figure 5, which is based on A schematic diagram of a face model selected in the manner of this embodiment.

The facial model material 6 includes a plurality of facial matching points 61. The face-to-face point 61 is preset in each face model data, and substantially the arrangement of the face-to-face points 61 forms the outline of the face model, as shown in FIG.

The server 3 transmits the face feature data and the face model data to the communication device 2 through the transmission unit 31. When the communication device 2 receives the two data by the transmission unit 21, the communication device 2 performs the following operations with the processing unit 23. FIG. 6 is a schematic diagram of the adjustment of the physical data according to the present embodiment. Referring to FIG. 6, first, the processing unit 23 uses the registration number relationship between the facial feature points 51 in the facial feature data and the feature region feature points 41 in the physical material data, according to the spatial coordinates of the respective facial feature points 51. The spatial coordinates of the feature points 41 of each feature area are respectively modified, and the result is that the arrangement of the feature points 41 of the feature area is changed, so that the pixel position of the body data is changed, so that the face of the element body, for example, but not limited to, the facial features are relatively The position, expression, etc., looks like the face of the flat avatar 5. In an aspect of the embodiment, the processing unit 23 first calculates a spatial coordinate difference between the facial feature point 51 and the feature area feature point 41 with the same registration number, and then uses a Radial basis function (RBF). The network, such a neural network-like software system, uses the difference to correct the physical data, so that the face of the body can appear very close to the plane avatar 5.

FIG. 7 is a schematic diagram of combining a face model with a meta-position according to the embodiment. Please refer to FIG. 7 and FIG. 3 at the same time, because the face-to-point 61 has its own registration number, and in addition, the physical data stored in the communication device 2 also has the elemental alignment point 71, and also has the registration number. Therefore, the processing unit 23 can combine the face model with the prime body according to the registration number relationship between the two points. The above steps are similar to the "skin" on the body, which is to be selected to have an approximate planar avatar 5 face features. The facial model is attached to the body, so that the body has the facial features of the planar avatar 5, such as, but not limited to, the width of the face, the width of the chin, and the like.

However, because the face of the body is the default standard face, there will be differences when the face model is combined with the body. For example, when the plane avatar 5 belongs to a narrow-faced chin, the facial model is also a narrow-faced chin. If the facial model is attached to the temperament, the displayed three-dimensional avatar will be accommodating at the cheeks. The body is convex, but there is a problem that there is a gap between the facial model at the chin and the body. At this time, the processing unit 23 needs to adjust the element body alignment point 71 of the element body according to the face-to-site point 61 of the face model. In this embodiment, the adjustment of the processing unit 23 is to change the spatial coordinates of the body-to-site point 71, and the pixel position when the body data is displayed is changed, so that the face model and the body are displayed together, There is no problem that the body is convex or there is a gap between the facial model and the body. Since adjusting the spatial coordinates of the element pair position 71 allows the element body to move closer to or away from the geometric center position, the effect is displayed as if the element body was removed or added.

Next, the processing unit 23 displays the physical data adjusted by the facial feature data and the facial model data together with the facial model data on the display unit, thereby generating a three-dimensional avatar corresponding to the planar avatar. Further, in the three-dimensional avatar displayed, the facial features, eyebrows, and the like are derived from the physical data adjusted by the facial feature data, and the facial skin covering the facial surface is derived from the facial model data. In the displayed three-dimensional avatar, the adjusted entity data and the face model data can be further combined by the processing unit 23 to display according to the combined single data; in addition, it is also possible to separate the two data forms. It is displayed by the processing unit 23 individually and displayed by maintaining the proper relative position of the pair of points, and the present creation is not limited thereto.

Of course, the above two steps of adjusting the physical data have no fixed sequence of execution. That is to say, it is also necessary to adjust the facial data type of the facial data with the facial model data, and then adjust the facial features with facial features.

In other embodiments of the present invention, the physical data may have only the upper body, the head, or even only the face, and the three-dimensional humanoid image that the user desires to generate includes a part other than the face.

In other embodiments of the present invention, the processing unit 23 of the communication device 2 may additionally map the three-dimensional avatar after the three-dimensional avatar is generated, so that the three-dimensional avatar has hair, glasses, beard or even other clothing accessories. Textures can also be achieved by means of site-assisted. Specifically, the three-dimensional avatar may have a matching position of the hair, and the hair module selected by the user also has a matching point, and the two opposing points are fitted in the space, that is, the space coordinates are the same, that is, The hair module can be combined with a three-dimensional avatar. Of course, other textures such as glasses, beards, etc. can also be made in this way.

In other embodiments of the present creation, the generated three-dimensional avatar can be combined with the set background to virtualize the feeling of the user's doll or doll in any place or environment; or, the three-dimensional avatar can be sent as a material for three-dimensional Print to produce physical dolls; or 3D avatars can also be made into electronic cards or stickers, this creation is not limited here.

In another embodiment of the present invention, after the plane avatar is uploaded to the server, the server may first perform noise reduction or skin beautifier processing on the plane avatar to make subsequent recognition more accurate, or The resulting 3D avatar is brighter and more beautiful.

The present invention further discloses a three-dimensional avatar generating device. The three-dimensional avatar generating device includes a A transmission unit, a storage unit, and a processing unit. The storage unit stores a piece of physical data in advance. The processing unit is electrically connected to the transmission unit and the storage unit, respectively. The processing unit adjusts the physical body data according to the facial feature data and the facial model data, and generates a three-dimensional avatar according to the facial model data and the adjusted physical body data. However, the technical content and implementation details of the three-dimensional avatar generating device are substantially the same as those of the foregoing three-dimensional avatar system. For reference, the foregoing is not described herein.

In summary, the use of remote or cloud processing to obtain a three-dimensional avatar must face the problem of excessive data transmission, resulting in a slow speed, but according to the creation of the three-dimensional avatar generation system and its communication device, production method, By pre-storing the physical data in the communication device, and then receiving the facial feature data and the facial model data for adjustment, thereby generating a three-dimensional avatar, the huge three-dimensional physical data can be prevented from affecting the transmission speed, thereby improving the efficiency of the three-dimensional avatar. Furthermore, through this creation, we can balance the problem that the local end hardware resources are insufficient to process 3D data at high speed, and the remote or cloud processing data transmission is too large, so that the virtual doll or doll can be more easily promoted and applied in different aspects. .

This creation provides a more efficient use of hardware resources relative to the form in which the production of a three-dimensional avatar is completely handed over to a communication device or server. In addition, although the download of the physical data takes time, it is easier for the user to agree that the first download of the application (APP) takes longer to install, rather than taking the time when the application is to be used, etc. Consumer mindset.

The above is intended to be illustrative only and not limiting. Any equivalent modifications or alterations to the spirit and scope of this creation shall be included in the scope of the appended patent application.

1‧‧‧3D avatar generation system

2‧‧‧Communication device

21, 31‧‧‧ transmission unit

22, 32‧‧‧ storage unit

23, 33‧ ‧ processing unit

3‧‧‧Server

Claims (11)

A three-dimensional avatar generating device includes: a transmitting unit; a storage unit pre-storing a physical volume; and a processing unit electrically connected to the transmitting unit and the storage unit, wherein the transmitting unit receives a face The feature data and a facial model data, the processing unit adjusts the physical body data according to the facial feature data and the facial model data, and generates a three-dimensional avatar according to the facial model data and the adjusted physical body data. The three-dimensional avatar generating device according to claim 1, wherein the phylogenetic data is from a server. The three-dimensional avatar generating device according to claim 1, wherein the facial feature data or the facial model data is obtained by a server according to a planar avatar, and the planar avatar corresponds to the three-dimensional avatar. The three-dimensional avatar generating device of claim 1, wherein the facial feature data comprises a plurality of facial feature points, the physical material includes at least one feature region, and the feature region includes a plurality of feature region feature points. The plurality of facial feature points are individually corresponding to the plurality of feature region feature points, and the processing unit adjusts spatial coordinates of the plurality of feature region feature points according to the plurality of facial feature points. The three-dimensional avatar generating device according to claim 1, wherein the facial model data includes a plurality of facial matching points, the primiparous data includes a plurality of morphological pair sites, and the plurality of facial aligning positions Point and the plurality of element body pairs are individually corresponding to The processing unit combines the facial model data with the physical data. The three-dimensional avatar generating device according to claim 1, wherein the processing unit changes a spatial coordinate of a part of the physical material according to the facial model data. A three-dimensional avatar generating system, comprising: a server; and at least one communication device, communicatively connected with the server, and pre-storing a physical body data from the server, the server transmitting a facial feature data and a The facial model data is sent to the communication device, and the communication device adjusts the physical body data according to the facial feature data and the facial model data, and the communication device generates a three-dimensional image according to the facial model data and the adjusted physical body data. Avatar. The three-dimensional avatar generating system of claim 7, wherein the facial feature data or the facial model data is obtained by the server according to a planar avatar, and the planar avatar corresponds to the three-dimensional avatar. The three-dimensional avatar generating system of claim 7, wherein the facial feature data comprises a plurality of facial feature points, the physical material includes at least one feature region, and the feature region includes a plurality of feature region feature points. The plurality of facial feature points are individually associated with the plurality of feature region feature points, and the communication device adjusts spatial coordinates of the plurality of feature region feature points according to the plurality of facial feature points. The three-dimensional avatar generating system according to claim 7, wherein the facial model data comprises a plurality of facial matching points, wherein the physical body data comprises a plurality of primordial locating points, and the plurality of facial aligning positions The point and the plurality of element body pairs are individually corresponding for the communication device to combine the face model data with the body data. The three-dimensional avatar generating system according to claim 7, wherein the communication device changes a space coordinate of a part of the temperament data according to the facial model data.