KR20190069750A - Enhancement of augmented reality using posit algorithm and 2d to 3d transform technique - Google Patents

Abstract

Disclosed is a method for representing augmented reality (AR) using a technique converting 2D into 3D and a pose from orthography and scaling with iterations (POSIT) algorithm, to provide smooth communication between a hairdresser and a customer. According to one embodiment of the present invention, a 3D face reconfiguration technique using 2D images, such as a facial image, is provided. Prior face knowledge or a generic face is used for detecting scare 3D information from the images and identifying image pairs. Bundle adjustment is performed for determining a more accurate position of a 3D camera, the image pairs are corrected, and high density 3D face information is detected without using the prior face knowledge. Outliers are removed by using tensor voting for instance. A 3D surface is detected from the high density 3D information and detail surface information is detected from the images. Coordinates of a body are figured out through the POSIT algorithm and thus a 3D object acquired therethrough can be represented thereon as AR.

Description

TECHNICAL FIELD [0001] The present invention relates to a technique for converting 2D to 3D and a method for representing augmented reality using POSIT algorithm. [0002]

The present invention relates to a technique of converting 2D to 3D and a method of representing augmented reality using the POSIT algorithm, and more particularly, a 3D shape file of a head is created using a technique of converting a 2D photograph to a 3D form. Then, POSIT (Pose from Orthography and Scaling with ITerations) algorithm is used to find the 3D object, that is, the pose of the person. .

Once acquired images are often processed based on prior knowledge or estimates of facial features that are generally visible, and conventional augmented reality hair styling techniques are methods of applying objects to previously imaged images.

With existing technologies, 3D objects can not be applied in real time through a camera, and passive optimization work may have to be accompanied.

Simulation of 3D hair style, which is a detailed and realistic 3D hair style created by common features using 2D images, is used to represent the 3D object on the coordinates obtained by real time POSIT algorithm through the camera.

The present invention is inconvenient that many people who are regularly hairdressers are unable to accurately determine whether or not a hair style of a desired model in a beauty salon matches with itself. Also, there are people who are not satisfied with hairdressers because they do not have the style they want. We simulate the more detailed and realistic 3D Hair Style to these people by giving them a virtual way to reduce these inconveniences and provide smooth communication between the hairdresser and the customer.

The technique of 2D to 3D conversion and the augmented reality technique using the POSIT algorithm includes analyzing a plurality of facial images to find sparse three dimensional facial shapes using prior knowledge of facial features, Using the rare three-dimensional facial shapes to analyze the plurality of images to find dense three-dimensional shapes using a data driven approach without using prior knowledge, And determining the pose of the moving body using the algorithm of Pose from Orthography and Scaling with ITerations and applying it to the coordinate value of the corresponding position.

According to the embodiment of the present invention, many people who are regularly hairdressers feel inconvenience that it is difficult to accurately determine whether or not a hair style of a desired model in a beauty salon matches with itself. Also, there are people who are not satisfied with hairdressers because they do not have the style they want. We can simulate the more detailed and realistic 3D Hair Style on the head of these people, thereby reducing the inconvenience and providing smooth communication between the hairdresser and the customer.

Figure 1 shows a flow diagram of the overall operation.
Figure 2 shows a general purpose computer capable of performing a flow chart.
3 shows a 3D conversion operation flow chart.

The present invention describes techniques for obtaining three-dimensional facial information using an assistive technique and describes a method of implementing an augmented reality with POSIT algorithm using it. According to aspects of the present invention, the prior knowledge of the facial structure is used at some points during the processing operation, but other parts during the processing operation are pure data driven.

Another operation uses a single camera for the determination of 3D animation from a set of 2D images.

The present invention is a method for analyzing a plurality of facial images in order to find sparse three-dimensional facial shapes using prior knowledge of facial features, (12) of using the rare three-dimensional facial shapes to analyze the plurality of images to find dense three-dimensional shapes using a data driven approach, and a step of performing a POS (Pose from Orthography and Scaling and a step 11 of recognizing the pose of the moving body using the algorithm with the ITERATION algorithm and applying the pose to the coordinate value of the corresponding position. Express.

The present application refers to determining three-dimensional information about an object, such as a face. Although embodiments of the present invention are described with reference to facial 3D reconstruction and rendering, it should be understood that this same technique can be used to reconstruct and render multiple views of any object. When used for a face, the three-dimensional information generated by the techniques described herein can be used for any face-based application such as animation, recognition, or rendering. The techniques described herein may be more realistic than other techniques that are more dependent upon the prior knowledge of the general facial.

The inventors of the present invention recognize that prior systems that used a strong prior knowledge of facial appearance to reconstruct the face actually quantize the number of basic shapes used to form and render facial features. A strong prior knowledge or general facial approach is effectively limited by the degrees of freedom provided by the prior facial knowledge or general facial imposed. So information and subsequent reconstruction does not capture all the fine details of the original face.

This "face space" quantization occurs because the prior knowledge and associated transforms limit the space of all possible faces that can be reconstructed by the system. General facial or pure prior facial knowledge-based methods may not have sufficient degrees of freedom to cover the entire facial space.

One embodiment is to use a data driven approach to disregard prior facial knowledge or general facial limitations at important points of the process and instead to find details of facial features referred to herein as dense features And captures fine facial detail by relying on data. The data-driven approach requires large amounts of data to effectively deal with noise, measurement uncertainty, and outliers. However, the system not only does not use a pure data driven approach, but is also assisted by methods that incorporate prior facial knowledge or common facial features.

According to an aspect of the invention, a large amount of data can be obtained from a single camera operating to obtain multiple images. For example, it can use frames of video that collectively form a moving sequence of images. This can also be obtained from multiple other still images obtained from one or more cameras.

One embodiment is described with reference to the flow chart of FIG. Figure 1 also illustrates some exemplary thumbnail images illustrating operation. This flowchart can be performed with any general purpose computer, such as the system shown in FIG. The system includes a processor 200, a user interface 205 such as a mouse and a keyboard, and a display screen 210. The computer may be, for example, an Intel based processor or any other type of processor. The computer receives raw or unprocessed video from one or more cameras 215, such as stillcamera or video cameras. The processor 200 processes the raw video data according to the description provided herein. Alternatively, the camera information may be stored in memory 220, such as a hard drive, and may be processed after a period of time.

One embodiment detects information from, for example, a video sequence, a sequence of still motion format images from a video sequence, or simply a sequence of images, such as multiple still images. If the subject does not stand still and the camera changes position, the sequence of images will have multiple different views of the subject's head in the set of images.

In step 100 of FIG. 3, the initial posture estimation is determined. It can use a face tracking algorithm to derive the initial head posture estimate and also to derive a mask that expresses the shape of the face. This uses the prior knowledge of the facial structure to determine the approximate location and posture of the head, the location of facial features such as the nose, mouth, etc., and the like. The mask helps to estimate the posture of the images.

The posture estimation technique delivers a set of views to a sparse feature tracking module at 110. Views transmitted to the module are those that are good candidates for image pairs for which a three-dimensional image can be detected. The sparse tracking module 110 generates a set of form matches for each image pair. The two images of the pair are sufficiently close, and thus these type matches can be obtained.

The posture selection is performed in step 120 and selects images that make the appropriate pair that can be used for the determination of the 3D information. These pairs should be close in attitude and have similar optical properties.

Global optimization is performed at step 130 on the entire set of shape points. This is used to improve camera position estimation and to calculate the three-dimensional structure of rare two-dimensional shapes.

The improved camera positions are used to modify the pairs of images in step 135, thereby limiting the search space for corresponding shape points to a horizontal scan line of the pair of images.

In step 140, dense feature matching is performed crossing the pairs. This finds additional forms beyond the rare detection that was performed in step 110. [ These correspondences can be judged by triangulation using camera postures optimized to form a high density 3D point cloud or mismatch map.

The point clouds corresponding to the individual pairs are merged into a single cloud, and the outliers are rejected in step 145. Dense feature detection is data driven entirely without using prior facial knowledge or common facial features. Step 150 defines dense feature computation aids that are used as simplifications to high density form matching. This includes ideal value rejection techniques (e.g., tensor voting) and may include area search minimization.

At step 155, a final cleaned point cloud is used to form the connected surface. The face texture is obtained from the frontal image. The final result is information representing the face. This is a 3-D mesh formed of triangular pieces. This final result may alternatively be a set of 3D points, or a plane defined by, for example, curved slabs, subdivided faces, or other digital surface definitions.

Additional details of the operation are now provided.

Conventional stereolithography schemes rely on the presence of multiple cameras to obtain one or more similar image pairs. The morphological correspondence between these multiple pairs of images is determined. The shape response can then be triangulated to find the final three-dimensional group of points.

In one embodiment, a single camera is used to obtain multiple images, after which the images are converted into multi-view stereoscopic images. In one embodiment, this process assumes that the head is stationary and that the camera is moving or moved relative to the head. While this is not likely, this assumption can provide a loss of generality, such as, for example, a camera may be static, head moved, or both the camera and head may be moved.

As described above, the multiple images are initially analyzed at step 100 to determine an initial estimate of the camera posture between the images. This initial estimate uses information representing facial features such as, for example, prior facial knowledge or a general facial, to perform the estimation. This may provide "sparse" information that allows the system to determine sufficient information to find the correspondence and attitude between the images.

For example, initial estimates performed with a prior facial knowledge or a general facial may provide information indicating a facial boundary, locations of the mask defining portions of the facial, or other information. This provides information for image selection and limits the set of matching rare shapes. Prior facial knowledge or general facial features are used to generate sparse shapes, but sparse shapes can be improved using data driven optimization prior to determining high density shapes.

The tracer attitude estimation module examines images to find similar images that can be modified for each other. Similar images contain images that define similar postures. This allows the selection of subsets of images to be used for reconstruction. These images are selected using both the baseline information as well as the reliable traced shape points that cross multiple images.

There is always uncertainty of measurement between multiple different images. For example, if the angular base line decreases between pairs of images, the error of the calculated 3-D points is magnified. This reduced angled base line thus increases the uncertainty of the 3-D measurement. Less accurate 3D information can be obtained from images with smaller angled base lines between images.

As the angled base line increases, more accurate 3D information can be detected, but there is also a smaller surface area that is common between the two views, thus matching with a smaller possibility. Thus, the image pairs are selected for balance between the number of measurement uncertainties and errors. For example, images with six points matched across an image pair and an angled base line of 8 to 15 degrees may be preferred.

Balancing can be performed by tracking the shape points of the multiple selection images. Only images with high confidence matches (e.g., greater than 90%) between forms are maintained to set up shape chains. The frame pairs are maintained in the set of images if they match the shape points and also meet the set base line criteria. For example, the baseline reference may be set to require an angled base line of at least 5 degrees. The shape point criterion also rejects frames with largely inaccurate tracer attitude estimates.

This sparse matching phase produces a set of images and shape points that cross the sequence and match. The matches provided by this shape point matching will be more accurate than the matches predicted alone by the posture tracer. Morphological point matches can also cover a larger number of frames than tracker predicted matches, thus providing even greater limitations on the camera attitude improvement process. These limitations can result in greater accuracy in posture improvement at step 130. [

A bundle adjustment begins with sets of matching points and images crossing the set of images. These are obtained by shape tracing as described above. The bundle adjustment performed in step 130 is an optimization technique for interpreting 3-D positions and camera parameters of points based on two-dimensional correspondences between sets of images. The optimized parameters may include the position and orientation of the camera and may include a 3-D structure of 2-D shaped points. The optimization can be performed by replacing the partial method for the structure and the partial method for the subsequent camera attitude.

Alternatively, the computer may perform all of these calculations until the appropriate method converges.

Then, the bundle adjustment estimates the position of the camera in each image by flip-flopping between estimating the position of the points and the position of the camera in an iterative manner until the final convergence. The end result is a more accurate camera position as well as the structure of the points. Because they are rare "high confidence" points, they do not provide a full dense representation, but are performed in subsequent stages.

An alternative technique may simply change values repeatedly until good values are obtained.

The 3-D positions of the matched morphologies as estimated and improved bundle adjustments 130 are used in subsequent stages to limit the extent of reconstruction. Which form optimized camera postures used in all subsequent processing stages.

The high density feature matching 140 finds more information about corresponding points between image pairs. However, unlimited high density matching can be computationally prohibited since it may require a full image search for each match. An unrestricted search will compare each point of each image for each point of every other image.

Step 150 represents techniques that are typically used to reduce the scope of high density form searching.

According to one embodiment, an epipolar geometry technique is used. In epipolar geometry, each corresponding item must be placed along a single line that extends between paired or densely packed images. The process can be further simplified by modifying the images, so that each epipolar line coincides with a horizontal scan line. This eliminates the need to re-sample images for each potential match.

After modification, corresponding points in each pair of images are found using a matching process. The prior facial knowledge or general facial features can be used to assist in the matching process by limiting matching to the areas covered by tracking the facial mask. This allows to simplify the search, so that a template is detected using a fixed window size for each pixel in one image. The template is matched along the corresponding epipolar line in the paired images.

The appropriate minimum correlation threshold and limited mismatch range for the face are used to reduce the number of false matches. Positions with a flat correlation plot are removed, and apparent peaks are not removed. However, multiple candidate matches can be maintained to find the best match.

The result of the matching process is the disparity volume. Each (x, y, d) triplet maps a pixel (x, y) of one modified image to a pixel (x + d, y) of the paired image.

Known attitudes can be triangular to convert discrepancy values into 3D points. Each mismatch pixel is transformed into its original image space using the inverse transform of the transform to modify. The three-dimensional position of the match is provided by the intersection between the rays passing through the optical center of the camera, and the corresponding pattern matches of the image plane. Indeed, the errors of shape matching and camera estimates will prevent these lines from crossing exactly. Three-dimensional points that minimize the orthogonal distance between rays can be used.

Other constraints may be provided by rejecting the outliers of the derived structure. The three-dimensional results from the bundle adjustment process provide a more accurate estimate of the three-dimensional facial structure even if it is rare. This is not enough to capture the facial microstructure. In one embodiment, this is used to provide a constraint on acceptable three-dimensional computation at high density reconstruction.

In particular, the calculated structure should not deviate from the structure derived by bundle adjustment. This structure is first used to prefilter data by transforming the internally inserted bundle-tuned structure into voxels and rejecting the data at a predetermined distance from the voxels. In practice, this becomes a data optimization technique.

The voxel check removes large anomalies farther than the predetermined distance from the bundle voxels. This also eliminates boundary artifacts due to incorrect placement of the facial mask. Errors in shape matching, however, can result in reconstruction noise. If noise is not correlated within views and between views, this will appear as a rare high frequency distortion in the three-dimensional structure. However, the exact matches will be correlated between views due to the continuity and smoothness of the facial structure.

Tensor voting can also be used to determine surface saliency, and thus maintaining a correlated structure is tensor voting. The 3D tensor voting method can be used to enhance and judge surface protrusion. Tensor voting allows each 3-D point to be encoded in either a ball tensor or a stick tensor. The information of the tensor is propagated to their neighbors through a voting operation. Neighbors have similar structures and thus reinforce each other through a tensor voting process. The amount of structural reinforcement is influenced by the initial structural protrusion. This technique restores the surface from the cloud of dots.

A good initial estimate of the point normals may be prioritized over the encoding of the points as empty tensors. In one embodiment, the head is approximated by a cylinder as shown in FIG. 4A. Cylindrical states are obtained. Cylindrical states can be used as point state approximations.

In one embodiment, the system may use a 3 x 3 EigenSystem and may fix the state as a first eigenvector in its native system. The remaining base vectors may then be computed using singular value decomposition. For example, the initial surface projection defined by the size difference between the first two eigenvectors can be uniformly set for all points.

<3D points obtained from bundle adjustment are very accurate, but are rare estimates of facial structure. These points are added to the tensor voting setting with the raised surface protrusion. Radial basis functions may also be used to interpolate a flat surface between the 3D points obtained from the bundle adjustment. In this embodiment, the states for the 3D bundle points are computed from the surface embedded therein for use for tensor voting. However, the internally inserted surface itself is preferably not used for tensile voting.

After two passes of the tensor bowl, the points with low surface protrusions are removed, leaving a dense cloud of dots scattered across the surface of the face.

Prior facial knowledge or general facial features may be introduced into the high-density reconstructive stage, and thus facial space is not limited. In particular, one embodiment may use the prior facial knowledge or general facial in a high-density process to determine and remove anomalies based on, for example, an approximation of an existing generic facial expression, but it is also possible to calculate or modify the 3D position of the reconstructed points It is not used for.

The facial detail structure is effectively captured in a three-dimensional point cloud. If the final goal is a mathematical description of the face, a 3D point cloud will suffice.

The POSIT algorithm can find out the coordinate value of the head of a human body among the body by using a method of inserting the virtual object into the camera using only the relative coordinates between the 3D objects.

If only the relative coordinates of the points of the 3D object are prepared, the camera matrix of each video frame can be calculated and synthesized considering the cloaking of the virtual object.

11: Coordinate value extraction using POSIT algorithm Step 12: Step of converting 2D image into 3D
20: Implementation step 100: Determination / estimation of attitude
110: Shape tracing 120: Posture selection
130: Posture improvement 135: Modification of images
140: High Density 2D Shape Matching 145: High Density 3D Reconstruction
150: High Density 3D Post Processing 155: Surface Reorganization and Organization
200: process 205: user interface
210: display 215: camera
220: Memory

Claims (41)

Analyzing a plurality of facial images to find sparse three-dimensional facial features using facial prior knowledge;
Using the rare three-dimensional facial shapes to analyze the plurality of images to find dense three-dimensional shapes using a data driven approach without using any prior knowledge; And
Identifying a pose of the body using a POSIT (Pose from Orthography and Scaling with ITerations) algorithm and combining the pose with the coordinate value of the corresponding position;
Containing
And a method for expressing augmented reality using POSIT algorithm. The method according to claim 1,
Wherein the analyzing the plurality of facial images comprises:
Using the prior knowledge to identify types in the images; And
Modifying between pairs of images to find pairs of similar images
More included
Wherein the facial reconstruction method comprises the steps of: The method according to claim 1,
Wherein the prior knowledge is used to identify portions of the face. The method according to claim 1,
Wherein the prior knowledge is used to identify a face mask representing a generic face. &Lt; RTI ID = 0.0 &gt; 11. &lt; / RTI &gt; The method according to claim 1,
Wherein using the prior knowledge comprises using the prior knowledge to limit facial shapes that form a set of rare three dimensional shapes. &Lt; Desc / Clms Page number 21 &gt; 3. The method of claim 2,
Wherein the pairs of similar images are pairs of images that are sufficient to identify the three-dimensional information but contain an angular baseline that does not increase beyond a certain amount to unintended increase in measurement uncertainty. &Lt; RTI ID = 0.0 &gt; . 3. The method of claim 2,
Characterized by further comprising the step of testing said image pairs to require an angular base line greater than a particular amount of a first amount and to require a correspondence between pairs of forms greater than a particular amount of a second amount A facial reconstruction method The method according to claim 1,
Analyzing the plurality of images a second time to find image clusters having feature point matches between images larger than a particular amount, and forming a set of traced shape points Further comprising using the image clusters to improve a first analysis performed using the prior knowledge in order to improve the first analysis. &Lt; Desc / Clms Page number 21 &gt; 9. The method of claim 8,
Wherein the image clusters comprise pairs of images. 9. The method of claim 8,
Further comprising using the set of traced shape points to find the position and movement of the traced shape points. &Lt; Desc / Clms Page number 13 &gt; 11. The method of claim 10,
Further comprising using the position of the traced shape points to improve the sparse three-dimensional shapes. &Lt; RTI ID = 0.0 &gt; 11. &lt; / RTI &gt; The method according to claim 1,
Wherein finding the high density features includes limiting a search range for the high density features. The method according to claim 1,
Wherein finding the high-density features includes rejecting outlier parts that are farther than a predetermined distance than other features. 14. The method of claim 13,
Wherein rejecting the ideal value portions comprises converting data into voxels and rejecting data that is greater than a predetermined distance from the voxels. The method according to claim 1,
Rejecting portions deviating from the surface protrusion by more than a specified amount. 16. The method of claim 15,
Wherein said rejecting step comprises retrieving said portions by using tensor voting. &Lt; Desc / Clms Page number 21 &gt; In a facial reconstruction system,
A camera for acquiring a plurality of facial images; And

CLAIMS What is claimed is: 1. A method, comprising: processing a plurality of images to find sparse three-dimensional facial shapes using facial prior knowledge as a processing part; Wherein the processing unit uses the rare three-dimensional facial shapes to analyze the plurality of images to find high-density features using a data driven approach. 18. The method of claim 17,
Wherein the camera is a still camera. 18. The method of claim 17,
Wherein the camera is a video camera. 18. The method of claim 17,
Wherein the processing unit is operative to use the prior knowledge to identify types in the images and to modify between pairs of images to find pairs of similar images. 21. The method of claim 20,
Wherein the processing unit uses the prior knowledge to identify a face mask representing a general face. 18. The method of claim 17,
Wherein the processing unit is operative to request the angular base line greater than the first specific amount and to test the image pairs to require a correspondence between the types of pairs greater than the second specific amount. system. 23. The method of claim 22,
RTI ID = 0.0 &gt; 1, &lt; / RTI &gt; wherein the image clusters comprise pairs of images. 18. The method of claim 17,
Wherein the processing unit finds dense forms by rejecting outlier parts that are greater than a predetermined distance from other forms. 25. The method of claim 24,
Wherein the processing unit performs the rejecting step using tensor voting. &Lt; Desc / Clms Page number 13 &gt; In a facial reconstruction method,
Analyzing the plurality of images from a single camera to modify the plurality of images and to find three-dimensional information representative of at least one face from the plurality of images, wherein analyzing the plurality of images An initial analysis using facial prior knowledge to determine the initial forms in the images and a subsequent analysis using the initial forms of the face to discover additional information without using any prior knowledge And analyzing the at least one facial image. 27. The method of claim 26,
Using the prior knowledge to identify the types of images, and modifying between pairs of images to find pairs of similar images. &Lt; Desc / Clms Page number 13 &gt; 27. The method of claim 26,
Wherein the prior knowledge is used to identify portions of the face. 29. The method of claim 28,
Wherein the prior knowledge is used to identify a facial mask representing a generic face. 28. The method of claim 27,
Wherein the pair of similar images are image pairs sufficient to identify the three-dimensional information but comprise an angular base line that is not large enough to inadvertently increase measurement uncertainty beyond a certain amount. 27. The method of claim 26,
Wherein the subsequent analysis comprises limiting the search range of high density features. 27. The method of claim 26,
Rejecting outlier portions that are farther than predetermined distances from the other features. &Lt; Desc / Clms Page number 22 &gt; 33. The method of claim 32,
Wherein rejecting the portion of the anomalies comprises converting data into voxels, and rejecting data farther than a predetermined distance from the voxels. 27. The method of claim 26,
Rejecting portions deviating from a surface protrusion greater than a specified amount. &Lt; Desc / Clms Page number 22 &gt; 35. The method of claim 34,
Wherein the rejecting step comprises searching for the portions using tensor voting. &Lt; RTI ID = 0.0 &gt; 11. &lt; / RTI &gt; Analyzing a plurality of facial images to find sparse information from the face; And

Using the sparse information to discover dense information using a data driven approach, the step of using the sparse information using a tensor voting technique, And limiting the high density information search range. In a facial processing method,
Analyzing a plurality of facial images to find matches between images using a prior knowledge of a generic face, the matches being used to generate sparse information; Analyzing the plurality of facial images to be used;

Using the matches to form pairs of images; And

Analyzing the pairs to find a set of dense forms using a data driven approach without any prior knowledge of the general face, Wherein analyzing the pairs comprises removing outlier portions from the set of dense forms. &Lt; Desc / Clms Page number 22 &gt; A method for automatically reconstructing a 3D face from a plurality of 2D images of a human face, the method comprising: using a generic face prior face knowledge to derive an initial camera position estimate;

Selecting pairs of images and detecting sparse shape points for each of the pairs of images;

Improving the original camera position estimate and the sparse shape points;
Using a pure data driven approach in detecting dense 3D point clouds from the image pairs;

Merging the high density 3D point clouds into a single 3D cloud;

Removing outliers from the single 3D point cloud to form a cleaned 3D point cloud;

Fitting the connected surface to the cleaned 3D point cloud; And

A method for automatically reconstructing a 3D face from a plurality of 2D images of a human face, comprising texture mapping the face detail information and color information of the face of the subject to the connected face CLAIMS What is claimed is: 1. A method for automatically reconstructing a 3D face from a plurality of 2D images of a human face, the method comprising: using a prior face knowledge of a generic face in deriving a first camera position estimate;

Selecting pairs of images and detecting sparse shape points for each of the pairs of images;

Improving the original camera position estimate and the sparse shape points;

Using a pure data-driven approach in detecting dense 3D point clouds from the image pairs;

Merging the high density 3D point clouds into a single 3D cloud;

Using a general facial prior art facial knowledge to remove outliers from the single 3D point cloud to form a cleaned 3D point cloud;

Fitting the connected surface to the cleaned 3D point cloud; And

And texture mapping the face detail information and color information of the face of the subject to the face. &Lt; RTI ID = 0.0 &gt; 11. &lt; / RTI &gt; The method according to claim 1,
The POSIT algorithm uses only relative coordinates between 3D objects to insert virtual objects into the camera
Using
How to find out the coordinate value of the human head among the fuselage. 41. The method of claim 40,
If only the relative coordinates of the points of the 3D object are prepared, the camera matrix of each video frame can be calculated and the synthetic method considering the cloaking of the virtual object.

Images