KR20010057718A - The face tracking system and method using 3 dimensional position controller - Google Patents

Abstract

PURPOSE: A face tracing system and method thereof using three dimensional position controller is provided to give a stable and good picture by keeping a position relation with camera and face using facing tracing apparatus. CONSTITUTION: A three dimensional position tracing measure(10) senses and traces motion of head in three dimensional space. A position and angle operator(20) calculates position and angle of user's head motion according to information about head motion of people traced by the three dimensional position tracing measure(10). A three dimensional position controller(30) is responded according to position and angle calculated by the position and angle operator(20), and then controls that a camera(40) adhered to the end of apparatus is positioned where has always a designated distance and angle against user's face. A picture display measure displays pictures which is taken a picture of by the camera(40).

Description

Face tracking system using 3D position controller and its method {The face tracking system and method using 3 dimensional position controller}

The present invention relates to a face tracking system and method for obtaining a face image of a fixed user, and in particular, face tracking for obtaining a fixed face image while maintaining face information such as a facial expression in spite of head movement. A system, a method thereof, and a computer-readable recording medium having recorded thereon a program for realizing the method.

A human face is a body part that has a very important meaning in mutual communication, and contains a lot of information. In other words, emotions may be transmitted from the facial expressions of the other party, and in the conversation, not only the voice but also the change in the shape of the other party's lips becomes important information for understanding the content of the conversation.

Due to this importance, many researches for using face information have recently been conducted in the field of multimedia technology. In particular, studies have been actively conducted to use such information in developing systems such as video conferencing or in developing computer graphics and virtual reality systems.

In this case, after obtaining an image of a face using a camera, it is preferable that the face is fixed in the image to obtain face information by processing the same.

However, due to human characteristics, frequent head movements and rotations occur. For this reason, the image photographed by the camera of the fixed position is difficult to obtain a fixed image of the face continuously, which is difficult to extract high quality facial information.

On the other hand, there is also a conventional method of using a helmet with a camera, in this case there is a problem that the facial expression of the user is not natural due to the burden of the helmet can not extract the quality of the face information.

The present invention has been made to solve the above problems, using a three-dimensional position controller to maintain a constant distance and angle with the camera at all times face tracking system and method and method for obtaining a fixed face image It is an object of the present invention to provide a computer-readable recording medium that records a program for realization.

That is, the present invention tracks the movement of the head using a three-dimensional position tracker, and after calculating the position and azimuth angle of the camera based on the movement, the three-dimensional position controller moves the camera according to the position and azimuth angle at all times. It is an object of the present invention to provide a face tracking system for obtaining a fixed face image by maintaining a constant distance and angle with a face, and a computer readable recording medium recording the method and a program for realizing the method.

1 is a block diagram of an embodiment of a face tracking system using a three-dimensional position controller according to the present invention.

2 is a flowchart illustrating a face tracking method using a 3D position controller according to the present invention;

* Explanation of symbols for the main parts of the drawings

10: 3D position tracker 20: position and azimuth calculator

30: 3D position controller 40: camera

50: video display unit

According to an aspect of the present invention, there is provided a face tracking system for obtaining a fixed face image, comprising: three-dimensional position tracking means for detecting and tracking a user's head movement in a three-dimensional space; Position and angle calculating means for obtaining a position and an angle at which the user's head is moved according to the head movement information of the person tracked by the three-dimensional position tracking means; And three-dimensional position control for reacting according to the position and angle obtained by the position and angle calculation means so that the photographing means attached to its terminal (end point) is always at a constant distance and angle with the user's face. It comprises a means.

In addition, the apparatus of the present invention, the image display means for displaying an image taken by the photographing means; Face information extracting means for extracting face information by receiving an image photographed by the photographing means; And video conferencing means for receiving a video photographed by the photographing means to conduct a video conference.

On the other hand, the method of the present invention, in the face tracking method applied to the face tracking system for obtaining a fixed face image, the first step of setting the positional relationship between the user face and the camera so that the captured face image is displayed properly ; Detecting and tracking the movement of the user's head; A third step of generating motion information by obtaining a new position and azimuth to be moved by the camera according to the motion tracked in the second step; And a fourth step of moving the camera in response to the motion information generated in the third step so that the distance and the direction relationship between the user face and the camera are kept constant.

The method of the present invention may further include a fifth step of extracting face information by receiving an image captured by the camera.

On the other hand, the present invention, in order to obtain a fixed face image, a face tracking system having a processor, the first function for setting the positional relationship between the user face and the camera so that the photographed face image is displayed properly; A second function of detecting and tracking a head movement of the user; A third function of generating motion information by obtaining a new position and azimuth to be moved by the camera according to the motion tracked by the second function; And a program for realizing a fourth function for moving the camera in response to the motion information generated by the third function so that the distance and direction relationship between the user face and the camera is kept constant. Provide a readable recording medium.

The above objects, features and advantages will become more apparent from the following detailed description taken in conjunction with the accompanying drawings. Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

First, in order to help the understanding of the present invention as described above look at the concept of the present invention in more detail as follows.

Faces are not only a means of recognizing people but they also play a very important role in people's communication. In other words, in addition to verbal conversations, ancillary things such as facial expressions and gestures have a very important meaning. In particular, facial expressions are used as a means of directly communicating the emotions of the other person. Due to this importance, the recently developed multimedia field has developed a video conferencing system and attempts to communicate using images of faces. In addition, with the development of computer graphics and virtual reality systems, humanoid three-dimensional characters appeared, and the importance of faces was emphasized in this character as well. In other words, facial expression animation has been developed in various forms such as mimicking the human facial structure.

However, these animations are only somewhat unnatural in generating facial expressions because they use only real human face structure information. In order to solve this problem, an attempt has been made to reflect the expression of a real person in facial animation, and continuous facial information is needed.

On the other hand, due to human characteristics, frequent head movements and rotations occur, which is an obstacle to continuous facial information extraction. That is, not only changes in facial expressions but also head movements should be considered. To this end, various researches have been conducted in the field of computer vision, but since the head has not only the movement but also the rotation, the face tracking is limited in the image obtained after fixing the position of the camera.

Therefore, in order to continuously obtain information such as facial expressions without being affected by the movement caused by the movement and rotation of the head, the face must be fixed to the image taken from the camera. For this purpose, it is necessary to fix the position between the face and the camera and maintain this positional relationship.

Accordingly, the present invention proposes a face tracking system that can obtain a fixed face image by using a 3D position tracker and a 3D position controller with a camera attached to a terminal (end point). It obtains head movements using a 3D position tracker, calculates the position and azimuth of the new camera based on this, and the 3D position controller moves the camera attached to the terminal to the position and angle, so that the fixed angle and distance Get a face image taken from.

That is, when the head moves, the movement and rotation information is sensed by the three-dimensional position tracker, and the position and azimuth calculation unit uses this information to obtain a new position and orientation of the camera. That is, in order to move the camera attached to the terminal of the three-dimensional position controller at the obtained position and angle, the position and azimuth calculation unit obtains the position and azimuth using inverse-kinematics, and the value is three-dimensional position. Applies to the controller. Therefore, even at the new position and angle, the camera maintains the same relationship as the distance and angle before the head moves, thereby obtaining a fixed face image from the captured image.

The three-dimensional position controller also helps the user move more naturally than fixing the camera to a helmet or other head. As a result, a user with less burden generates a natural facial expression, thereby obtaining quality facial information.

1 is a configuration diagram of a face tracking system using a three-dimensional position controller according to the present invention, a three-dimensional position tracker 10, a position and azimuth calculator 20, a three-dimensional position controller 30, three-dimensional A camera 40 attached to the position controller 30. In addition to this, an image display unit 50 may be further provided for displaying a captured image, and a system for video conferencing (not shown) may be further connected.

Looking at this in detail, as shown in the figure, the present invention tracks the user's head movement (movement and rotation) using the three-dimensional position tracker 10 in the three-dimensional space.

Then, the position and azimuth calculation unit 20 calculates a new position and orientation of the camera using inverse-kinematics based on the head movement information of the person tracked by the 3D position tracker 10. .

Then, the 3D position controller 30 reacts according to the new position and azimuth angle obtained by the position and azimuth calculation unit 20, so that the camera 40 attached to the terminal (end point) always maintains a constant distance and angle with the face. Positioned so that the camera 40 can obtain a fixed face information from the image taken by the camera 40.

Meanwhile, a face information extraction unit (not shown) for receiving the image captured by the camera 40 and extracting face information is connected to the camera 40 instead of the image display unit 50 or the image display. It may be configured by connecting to the camera 40 in parallel with the unit 50. The extracted face information can be used for a humanoid three-dimensional character used in computer graphics or a virtual reality system.

2 is a flowchart illustrating a face tracking method using a 3D position controller according to the present invention.

First, the 3D position tracker 10 is attached to the user's head and body, and the camera 40 attached to the terminal (end) of the 3D position controller 30 is set by setting the 3D position controller 30. The face image is displayed on the image display unit 50 as appropriate (201). When the user moves his head in this state, the motion information is detected by the 3D position tracker 10 and transmitted to the position and azimuth calculator 20 (202).

Then, the position and azimuth calculation unit 20 obtains a new position and azimuth of the camera in order to maintain a constant distance and direction relationship between the face and the camera (203), and sends the actual camera at this position and angle, in three dimensions. The motion information of the position controller 30 is generated and transmitted to the 3D position controller 30 (204).

Then, the 3D position controller 30 moves in response to the motion information received from the position and azimuth calculation unit 20, and as a result, the camera attached to the terminal of the 3D position controller 30 is moved to a new position and angle. Will move. Therefore, the position and direction of the face in the image displayed on the image display unit 50 are the same as before the head moves.

The face information extracting unit may further perform a process 206 of extracting face information by receiving an image photographed by the camera 40.

The present invention described above is not limited to the above-described embodiments and the accompanying drawings, and various substitutions, modifications, and changes can be made in the art without departing from the technical spirit of the present invention. It will be apparent to those of ordinary knowledge.

The present invention as described above, because the positional relationship between the camera and the face is maintained in spite of frequent movements of the head, it is possible to take a picture of a fixed face to extract more face information, such a face on the screen This fixed image has the effect of providing a more stable image when multimedia video conferencing or video telephony.

In addition, in the present invention, only a three-dimensional position tracker is attached to the head, so there is no burden, so that a natural facial expression can be generated, and information such as a higher quality facial expression can be extracted. If applied to the face of the virtual character used in the virtual reality system, more realistic animation is possible effect.

Claims (8)

In the face tracking system for obtaining a fixed face image, Three-dimensional position tracking means for detecting and tracking a user's head movement in a three-dimensional space; Position and angle calculating means for obtaining a position and an angle at which the user's head is moved according to the head movement information of the person tracked by the three-dimensional position tracking means; And 3D position control means for reacting according to the position and angle obtained by the position and angle calculation means so that the photographing means attached to its terminal (end point) is always at a constant distance and angle with the face of the user. Face tracking system comprising. The method of claim 1, Image display means for displaying an image photographed by the photographing means Face tracking system including more. The method according to claim 1 or 2, Face information extracting means for extracting face information by receiving an image captured by the photographing means Face tracking system including more. The method of claim 3, wherein Video conferencing means for receiving video captured by the photographing means and conducting a video conference Face tracking system including more. The method of claim 3, wherein The position and angle calculation means, And a new position and an orientation of the photographing means using inverse-kinematics based on the head movement information of the person tracked by the three-dimensional position tracking means. In the face tracking method applied to the face tracking system for obtaining a fixed face image, A first step of setting a positional relationship between the user face and the camera so that the photographed face image is displayed properly; Detecting and tracking the movement of the user's head; A third step of generating motion information by obtaining a new position and azimuth to be moved by the camera according to the motion tracked in the second step; And A fourth step of moving the camera in response to the motion information generated in the third step to maintain a constant distance and direction relationship between the user face and the camera; Face tracking method comprising a. The method of claim 6, A fifth step of extracting face information by receiving an image captured by the camera Face tracking method comprising more. In order to obtain a fixed face image, in the face tracking system with a processor, A first function of setting a positional relationship between the user face and the camera so that the photographed face image is displayed properly; A second function of detecting and tracking a head movement of the user; A third function of generating motion information by obtaining a new position and azimuth to be moved by the camera according to the motion tracked by the second function; And A fourth function of moving the camera in response to the motion information generated by the third function to maintain a constant distance and direction relationship between the user face and the camera; A computer-readable recording medium having recorded thereon a program for realizing this.