KR20020044750A - Head pose tracking method using image processing - Google Patents

Abstract

PURPOSE: A method of tracking a position and a direction of the head using an image is provided to track a 3-dimensional position and a direction of a computer user's head in real time by using images, and to supply an algorithm calculating the position and the direction by recognizing markers as a standard, thereby controlling interfaces between a human being and a computer with eyes. CONSTITUTION: A tracking system independently compensates the center of an image(S100). The tracking system fixes a camera(S110). The tracking system finds a conversion matrix presenting that markers are converted into coordinates of a computer image from real-world coordinates(S120). The tracking system finds an internal parameter of the camera from the conversion matrix(S130). The tracking system finds an external parameter from the conversion matrix(S140).

Description

How to track the position and direction of the head using video {HEAD POSE TRACKING METHOD USING IMAGE PROCESSING}

The present invention relates to a method for tracking the position and direction of a head using an image. More specifically, the present invention relates to a head motion tracking method that enables a practical application of a system capable of manipulating a human-computer interaction interface using eye gaze and intentional blinking.

Recently, as computer performance is improved, the interest of computer researchers is focused on the interface between humans and computers. Human / computer interface is a method of dialogue between user and computer, and research on this is being actively conducted worldwide. In particular, one of the purposes of human / computer interface research is to increase the communication bandwidth between the user and the computer.

Reflecting this trend, a human / computer interface using a hand or a head movement, a voice, or a pupil movement has recently been proposed. These studies did not consider head movements at all or fully, so that the user could control the computer only with the head fixed.

Currently, a system for tracking the position and direction of the head by an electromagnetic method has been proposed to ensure the free movement of the user's head. In other words, there are mechanical methods, electromagnetic methods, and optical methods to track the position and direction of a moving object. Among them, electromagnetic and optical methods are the most common.

The electromagnetic tracking method is high accuracy, but the distorted information can be obtained by interference from the surrounding magnetic field, and the system is complicated and expensive. On the other hand, the optical tracking method has a disadvantage in that information can be obtained only in a limited area (the visual field of the camera), but it is practical because the system is simple and inexpensive without the influence of the magnetic field.

In recent years, the development of high resolution and small size CCD (Charge Coupled Device) cameras has made it easier to attach cameras to headbands, and real-time image processing is possible with the development of computer vision technology.

By using the vision technology of the computer, it is possible to control the computer with the movement of the head for the handicapped or handicapped workers who are busy enough to use both hands. Can be.

However, in the conventional system for tracking the movement of the head, the magnetic field was used as an implementation means of the tracking system, but the distorted data was detected due to the influence of the surrounding magnetic field and ferrous metal. Thus, the reliability of the system is lowered.

SUMMARY OF THE INVENTION The present invention has been made to solve the above problems, and an object of the present invention is to 1) track the three-dimensional position and orientation of the computer user's head in real time using an image; 2) The aim is to provide an algorithm that calculates the position and direction of the head by recognizing markers as reference points.

As a technical idea for achieving the above object of the present invention

A first step of independently correcting the center of the image;

A second step of fixing the camera;

Obtaining a transformation matrix representing a relationship in which the marker is converted from real world coordinates to coordinates of a computer image;

A fourth step of obtaining internal parameters of the camera from the conversion matrix;

A method of tracking the position and direction of a head using an image comprising a fifth step of obtaining an external parameter from the transformation matrix is provided.

1 is a schematic diagram illustrating a system for tracking a position and a direction of a head by attaching a CCD camera to a user's head according to an embodiment of the present invention.

2 is a flowchart showing a process of tracking the position and direction of the CCD camera in the tracking system according to the present invention.

3 is a schematic diagram of a system for tracking the position and orientation of a head by attaching a marker to a user's head in accordance with another embodiment of the present invention.

<Description of the symbols for the main parts of the drawings>

10: infrared generator 20: CCD camera

30: image processing module 40: computer device

40a-40h: marker 50: glasses

50a-40c: Marker

Hereinafter, with reference to the accompanying drawings, the configuration and operation of the embodiment of the present invention will be described in detail.

Figure 1 schematically shows a system for tracking the position and direction of the head by attaching a CCD camera to the user's head in accordance with an embodiment of the present invention.

1, an infrared ray generator 10 for generating infrared rays in a forward direction within a range including a predetermined angle, a CCD camera 20 for capturing an image of a marker attached to a user's head and mounted on a monitor; And an image processing module 30 which is embedded in the computer device and processes the image of the photographed marker, and a computer device 40 that tracks and calculates the user's gaze movement, that is, the three-dimensional position and direction of the head. have.

Looking at the operation of the system, the image of the eight markers (40a to 40h) attached to the monitor of the computer device 40 attached to the headband on the user's head and attached to the monitor of the computer device 40 to obtain three images of the CCD camera 20 We get the dimensional position and coordinates, that is, the three-dimensional position and coordinates of the head.

That is, when the user gazes at the movement of the user's gaze and stares at a particular menu on the monitor screen of the computer device 40, the cursor is placed in the menu, and when the user blinks, the eye selecting the menu is blinked. In order to make the human / computer interface controlled by the robot more practical, the image processing technique is used as the movement of the head.

Here, the movement of the gazeline is calculated by a combination of the movement of the head and the movement of the pupil, and can track the movement of the head in real time. That is, the 3D position and direction of the user's head can be tracked in real time using the images.

In order to find out the three-dimensional position and direction of the head by using the image in real time, the image processing must be simply performed using the marker. We need an algorithm that recognizes markers as reference points and calculates the position and orientation of the head.

2 is an algorithm illustrating a tracking process of a position and a direction of a CCD camera in a tracking system according to the present invention.

Independently correcting the center of the image (S100);

Fixing the camera (S110);

Obtaining a transformation matrix representing a relationship in which the marker is converted from real world coordinates to coordinates of a computer image (S120);

Obtaining internal parameters of the camera from the transformation matrix (S130);

In step S140, external parameters are obtained from the transform matrix.

3 is a schematic diagram of a system for tracking the position and orientation of a head by attaching a marker to a user's glasses in accordance with another embodiment of the present invention.

Referring to FIG. 3, an infrared ray generator 10 for generating infrared rays forward in a range including a predetermined angle, a CCD camera 20 attached to an upper end of a monitor of a computer device and capturing an image of a user marker; An image processing module 30, which is embedded in a computer device and processes the image of the captured marker, a computer device 40 that tracks and calculates the movement of the gaze of the user, and three markers 50a and 50b as correction points. , 50c is composed of the user's glasses 50 to be attached.

Here, referring to FIG. 3, the CCD camera 20 is attached to the user's head in FIG. 1, but FIG. 3 is attached to the top of the monitor of the computer device 40. In addition, eight markers are attached to the monitor of the computer device 40 in FIG. 1, but three markers are attached to the user's glasses 50 as correction points.

Looking at the operation of the system, as shown in Figure 1, the practical value is simpler and more convenient than the method of attaching the CCD camera 20 to the head. The CCD camera 20 attached to the top of the monitor takes images of three markers attached to the glasses.

That is, the image processing module 30 embedded in the computer device 40 discretizes the image and obtains image coordinates of the center of gravity of the marker, and the computer device 40 has a three-dimensional position and orientation of the head (6 degree of freedom). ) Is calculated.

As described above, since the absolute coordinate of the gazeline in the present invention is determined by combining the movement of the pupil and the movement of the head and body, it is important to track the movement of the headline in real time. Therefore, ensuring free movement of the user's head is very important for a more natural and flexible use environment.

As described above, according to the method for tracking the position and direction of the head using the image according to the present invention, the three-dimensional position and direction of the head of the computer user can be tracked in real time using the image, respectively, and the markers as reference points. An algorithm that recognizes and calculates the position and direction of the head is provided so that it can be applied to virtual reality technology as well as the human / computer interface that adjusts by eye.

Claims (2)

A method of tracking a relative three-dimensional position and orientation of a head relative to a monitor screen by attaching a CCD camera to the head and attaching a marker to the monitor of the computer device, Independently correcting the center of the image; Fixing the CCD camera; Obtaining a transformation matrix representing a relationship in which the marker is converted from real world coordinates to coordinates of a computer image; Obtaining internal parameters of a CCD camera from the conversion matrix; A method for tracking the position and direction of the head using an image comprising the step of obtaining an external parameter from the transformation matrix. The method of claim 1, wherein the method for tracking the position and orientation of the head in a relative three-dimensional manner is characterized by tracking the position and orientation of the head with a marker attached to the user's glasses and a CCD camera attached to the top of the monitor. How to track the position and direction of the head using the image.