KR100215202B1 - Target tracking method and device for video phone - Google Patents

Abstract

[0001] The present invention relates to a tracking and tracking device for moving a camera by calculating an upper plate figure by arbitrarily determining an upper plate area in various shapes in order to track a target track in a video telephone.

In the present invention, instead of the conventional square window, the shape of an effective window is determined through an initial image processing process to estimate a top view, so that the calculation amount can be improved while improving tracking performance for tracking a target.

Description

A method and device for tracking a video telephone (Target TracKing Method and Device for Video Phone)

The present invention relates to a method and an apparatus for tracking a target by moving a camera by calculating an upper plate figure by arbitrarily setting a shape of an upper plate area effective for tracking a target in a videophone.

In general, tracking methods for tracking an object, that is, a target, include a center point tracking method and a top board tracking method.

As shown in FIG. 1A, the center-point tracking method is a method of tracking a center point (A) of a moving object separated from a background of a moving object.

At this time, the threshold value is used to separate the moving object, that is, the target, from the background.

In other words, the background and object are binarized by the threshold value.

However, this central tracking mechanism has a disadvantage that it is highly affected by noise.

In other words, if the image is relatively simple and the image region is reduced, the tracking stability is good and the effect of noise is small when the constraint on the speed of the trackable object is relatively small. On the other hand, if the image is relatively complex and the constraint on the speed of the object to which the image segmentation is applied is relatively large, the tracking stability is poor and the noise is increased.

As shown in FIG. 1B, the top plate tracking method is a method of detecting a moving object of a previous frame, that is, a region B having a size suitable for a position of a target, and a top plate (B '), which is the highest area of the top plate.

That is, the top plate tracking method defines a window region of a fixed size including a moving object, that is, a top plate region when a position of a moving object is given in a given nth image, And the position of the region having the highest correlation is calculated as the position of the moving object in the (n + 1) -th image.

Here, the shape of the initial window in the calculation of the top plate, that is, the top plate of the current frame and the previous frame, is mainly formed in the form of a square window.

Therefore, since the top tracking method calculates the correlation directly from the image of the current frame input without using the image segmentation process, the tracking performance is maintained for a relatively complicated image, but the calculation amount is increased.

In other words, the correlation tracking method generally does not binarize the image compared to the center-point tracking method. Therefore, it can be expected that the tracking performance can be expected even if the image regionization is not possible because background scattering is added to the image. In order to estimate the occupation, there is a disadvantage that the calculation amount is increased because the upper plate figure must be calculated.

In order to improve the above disadvantages, the present invention relates to a method of tracking a target telephone set for setting an area having a similar brightness value from an initial input image to a top plate area, calculating a top plate figure, And an object of the present invention is to provide an apparatus.

1A is a view for explaining a conventional center point tracking technique;

1B is a view for explaining a conventional top plate tracking technique;

2 is a flowchart of a target tracking method according to the present invention

3 is a detailed flowchart of the window waxing step and the shape determining step of Fig. 2

FIG. 4A is a view for explaining the window waxing step of FIG. 3; FIG.

Fig. 4B is a view for explaining the shape determination step of Fig. 3

5 is a diagram showing the configuration of a target tracking apparatus according to the present invention

DESCRIPTION OF THE REFERENCE NUMERALS

300: camera 400: motor part

410: motor driver 420: motor

500: control unit 600: image storage unit

610: A / D converter 620: current video low window

630: previous image storage unit 710: window setting unit

720: shifter 730: top plate calculation section

740:

In order to accomplish the above object, according to the present invention, there is provided a tracking method of a target object of a video telephone, comprising the steps of: determining a shape of a top plate region by determining a brightness value of an initial window and neighboring pixels adjacent to the initial window; Calculating a degree of correlation between a previous image and a current image input according to the shape of the upper panel region; estimating the euphony's position according to the calculated degree of correlation; And a call termination step of performing the correlation calculation step and the motion estimation and motor driving step with respect to an image input until the end of the call.

According to another aspect of the present invention, there is provided an apparatus for tracking a target object of a video telephone, the apparatus comprising: a video window means for A / D-converting an input video signal in a tracking mode; A window setting unit for determining the shape of the upper plate area by comparing brightness values of pixels with pixels of the upper plate area and the search area from the previous image and the current image output from the image storing unit according to the shape of the window set by the window setting unit And a motion estimating means for estimating a motion of the user based on the calculated top plan calculated by the calculating means, and a motion estimating means for estimating motion estimation based on the motion estimating means, And control means for controlling the camera to move in accordance with the operation.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

As shown in FIG. 2, the method for tracking a target telephone of a videophone according to the present invention includes steps of determining top plate shape (100, 101, 102, 103, 104, 105), top plate calculation steps (106, 107), tone estimation and motor driving steps (108, 109) Is performed by steps 110 and 111.

The step of determining the shape of the correlation area (100, 101, 102, 103, 104, 105) is a step of determining the shape of the upper plate area by comparing the brightness values of the initial window and the neighboring pixels in the initial image input in the tracking mode, An initial window operation step (102) for setting an initial window at a position of a target in the stored initial image (102), a brightness of an outermost pixel in the set window (102) (103,104) for setting a specific value according to a magnitude of a difference value between a zenith value of a predetermined window and a brightness value of a neighboring pixel adjacent to the set window to wake up the window, And a shape determination step 105 for determining a shape.

Here, the initial image is an image taken by a camera at a predetermined position, and the initial window is composed of 3 x 3 pixels.

The upper stage calculation step 106, 107 calculates a top view of a previous image and a current image input according to the shape of the determined correlation area.

Here, the correlation function for calculating the top plate diagram is NA, Nean Absolute Err.

The motion estimating and motor driving steps 108 and 109 are steps for estimating a darkness according to the calculated correlation and moving the camera.

The call termination steps 110 and 111 correspond to the steps of calculating the top board 106 and 107 and the motion estimation and motor driving steps 108 and 109 with respect to an image input until the call ends.

Hereinafter, the progress of the tracking method according to the present invention will be described in detail.

First, the shape of the upper plate area for tracking is determined by performing the upper plate area shape determination steps (100, 101, 102, 103, 104, and 105).

Since there may be cases where the user does not want to use the tracking function according to the leader of the call after the telephone is started, the user first determines whether the user uses the tracking function (100). The distinction between trace mode and non-trace mode can be implemented simply with a switch. That is, if the switch is turned ON, it will not be executed if the tracking function is off.

When the switch is turned on to perform a tracking function, an initial image storage step (100,101) for converting an initial image input from the camera into an A / D (Analog / Digital) is performed.

That is, the NTSC signal coming from the camera is output as an 8-bit digital signal through the A / D converter. At this time, a high-bit A / D converter can be used. Since this digital signal is an image signal, it is stored in the memory as a digital image I (x, y) having a value within a specified range on the secondary order matrix.

This digital image is an image with a size of 64 x 64 pixels and is stored in memory and used for target tracking.

In the top tracking, it is assumed that the initial position, that is, the position where the target caller can press the dial to make a call, is determined by the LocKing process of the initial target. Therefore, in the initial position, the camera can take the initial image when the target is in the position set to 1 for the call.

An initial window setting step 102 is performed to set a window composed of a square of 3 x 3 pixels, which can include a target person's face, as an initial window from the initial image thus taken.

That is, as shown in FIG. 4A, an initial window including a key cue of a speaker is set to a size of 3 x 3 pixels from an initial image of 64 x 64 pixels.

The initial window 102 is set to a specific value according to the size of the difference value between the brightness value of the outermost pixels in the set initial window and the neighboring pixels adjacent to the set window, The shape determination step 105 is performed to determine the shape of the upper plate region according to the waxing steps 103 and 104 and the set values of the ridge setting. This process will be described in detail with reference to FIGS. 3, 4A and 4B.

The image of the set initial window is a form in which the brightness value is assigned to the second-order matrix di x, y).

As shown in FIG. 3, the window waxing steps 103 and 104 include a pincushion value calculation step 201 for calculating a pincushion value of brightness values of the outermost pixels of the set window, a brightness value calculation step (202, 203, 204, 205) for comparing the difference value with a predetermined threshold to set a specific value, and if it is determined that the pixel having the fixed value is adjacent to the set window, And proceeds to the average value calculation step (201) when the pixels adjacent to each pixel of the set window are displayed as a window.

As shown in FIG. 4A, first, an initial window composed of 3 x 3 pixels (FO to F8) is set (102, 200), and a positive value of the brightness value of the outermost pixels of the set initial window is calculated (201). That is, as shown in FIG. 4A, the pixel value N of the brightness values of the outermost pixels F1 to F8 of the initial window is obtained (201) as shown in the following equation (1).

N = 1/8? I (x, y) ----- [Expression 1]

Where x, y) represents the brightness value of the pixel.

As shown in the above equation (1), the brightness of the outermost pixels in the initial window

(202), which is adjacent to the initial window in which the pixel value (N)

The brightness values of the pixels are retrieved (203).

That is, the average value of the brightness values is subtracted from the brightness value of the selected pixel, and the absolute value is taken to compute the absolute value of the brightness value (203). At this time, if the value obtained by subtracting the absolute value of the brightness value from the brightness value of the selected pixel as a result of the beg is not larger than the set threshold value, '1' is set to the selected pixel.

In other words, the differences between the pixel values N of the brightness values of the eight outermost pixels F1 to F8 of the initial window and the brightness values of the sixteen pixels F9 to F24 adjacent to the initial window are calculated, (204). If the difference is not greater than the threshold value, a value of '1' is set (204) in the corresponding pixel, and a comparison of the brightness values of the eight outermost pixels F1 to F8 It is retrieved (205).

On the other hand, if the difference is larger than the threshold value, '1' is not set, and the process proceeds to step 205 to search whether the comparison of the brightness values of the eight outermost pixels F1 to F8 is completed.

If the comparison of the brightness values of the eight pixels is not completed, the process proceeds to step 201 where a pixel adjacent to each pixel of the initial window is selected.

If the brightness of the eight pixels has been completed, a search is made (206) for any one of the pixels F9 through F24 adjacent to the currently set window, that is, the initial window, to which '1' is set.

At this time, if there is at least one pixel in which '1' is set, the window is set as a window by including a portion set to '1' in the currently set window, that is, the pixels F9 to F24 adjacent to the initial window (207).

That is, if there is at least one pixel in which '1' is set in the pixels F9 to F24 adjacent to the initial window, a region including pixels '1' among the pixels F9 to F24 adjacent to each pixel in the set initial window is included Let the window wax (207).

In the case where the window has been set as described above, the flow proceeds to step 201 for calculating the fingertip of the brightness value of the pixel whose inner pixel is '1' among the outermost pixels F9 to F24 of the set window.

That is, the pixel value of the brightness value of '1' among the outermost pixels (F9 to F24) of the waxed window is calculated (201), and the pixel value of the calculated brightness value and pixels adjacent to the pixel of '1' The difference between the brightness values of the pixels F25 to F48 is calculated 202 and 203, respectively, and the differences are respectively compared with the threshold values 204. If the difference is not greater than the threshold value, '1' It is determined whether the comparison of the brightness values of the sixteen outermost pixels F9 to F24 is completed (205).

On the other hand, if it is not determined that the pixel having the set '1' is adjacent to the set window, the process proceeds to the shape determination step 208 and the area including the pixel in which '1' is set as shown in FIG. And shape determination steps 105 and 208 are performed.

That is, a portion (di x, y) = 1 of pixels in which '1' is set is a portion of brightness at an initial position, which is somewhat uniform, initially corresponding to the upper plate region, . Since this shape differs according to the shape of the talker, it shows better performance compared to the initial window of the square hinge like the conventional method.

In this way, the upper plate is subjected to the calculation steps 106 and 107 for determining the shape of the upper plate region.

First, A / D (Analog / Digital) conversion of the input current image is performed and stored (106).

In the case of detecting the shape of the upper plate region for the first time in the initial window, since the previous image is the initial image, the region set to '1' is set as the upper plate region. This is because the portion of the previous image that has the highest degree of correlation is the top region, which is set to '1' in the initial image.

In step 107, a top area is extracted from the stored current image, and a top area between the top area and the top area is calculated.

In order to calculate the correlation between the top plate area and the search area, a top plate function to be selected must be determined.

NCCF (Normalized Cross Correlation Function), NS (Nean Square Error), and NA (Nean Absolute Error) are the top plate functions for calculating the top plate.

NCCF (p, q) = [In In (i, j) In + 1 (i + p, j + q)] /

(? In + 12 (i + p, j + q)) 1/2]

(P, q) = E (1 In (i, j) - In (i, j) +1 (i + p, j + q) 1)

Here, E (·) represents 1 average.

NCCF is most suitable as the top plate function, but NAE is used in the present invention by drawing the calculation amount.

The higher the NAE, the lower the value. Therefore, when the NAE is applied to the top plate function, the position with the lowest value is the moving object in the next image, that is, the estimated position of the target.

According to the function of the top plate, the top view of the current image and the top view of the top image of the initial image, which is the previous image, are calculated, and the portion with the highest correlation is the target of the search area, And the difference between the current position and the euphoria is calculated.

In other words, the difference between the position of the correlation area of the previous image, which is the previous image, and the position of the highest part of the top view in the search area is calculated to estimate the movement of the target face of the talker (108).

The motor is driven according to the estimated movement, that is, the talker moves to move the camera (109).

At this time, if a new image is input from the camera, a new image is inputted and then an image is stored (step 106).

The video signal of one frame is shifted to the next memory which is stored in one memory and stored as a previous video. That is, the video signals of the current video and the previous video are stored in the two memories, respectively.

Accordingly, the image stored in the image storage step 106 is the current image to be currently input, and the previous image including the initial image is shifted and stored in another memory for the next motion estimation.

When the video signals of the previous video and the current video are stored in the memory, the correlation area and the search area are saved.

That is, the uppermost portion of the upper panel subtracted in the motion estimation step 108, which is the highest portion of the upper panel extracted from the previous image, is extracted into the shape of the determined upper panel region, And sets the search area in the current image to the center of the upper plate area of the set previous image.

The upper stage calculating the top plate in the course of the calculation is carried out as described above in the calculation step 107, the township presumption step 108 and the motor driving step 109 as described above.

When a new image is not input from the camera, the call is terminated and the call is terminated.

That is, when a new image is input, the process proceeds to the image storage step 106 and the above process is repeated. When the new image is not input, the tracking function is terminated (110, 111).

Also, if a new video is not inputted after the call is terminated, the terminal waits until a new video is input.

5 is a block diagram showing the configuration of the image phone 600 according to the present invention. The image phone 600 includes a video storage unit 600, a motor unit 400, a control unit 500, a window setting unit 710, a shifter 720, ), And a motion estimation unit 740.

The image storage unit 600 A / D converts the image input in the tracking mode and stores the image. The A / D converter 610 performs A / D conversion on the image input in the tracking mode, the A / D converter 610 A current image storage unit 620 for storing a current image signal output from the current image storage unit 620 and a previous image storage unit 630 for storing a video signal output from the current image storage unit 620.

The window setting unit 710 determines the shape of the upper plate area by subtracting the brightness value of neighboring pixels adjacent to the window from the initial image that is stored in the image storage unit 600. [

The top plate calculation unit 730 calculates the top plate degree by extracting the top plate region and the search region from the previous image and the current image output from the image storage unit 600 according to the shape of the window set in the window setting unit 710 .

The euphonial job predicting unit 740 estimates that the upper board also has a low rank according to the degree of correlation calculated by the calculating unit 730.

The control unit 500 controls the operation of the image storage unit 600 and controls the camera to move according to the estimated soundness of the euphony position estimation unit 740.

The shifter 720 shifts the previous image stored in the image storage unit 630 and outputs the shifted image to the correlation calculation unit 730.

The motor unit 400 includes a motor driver 410 for driving the motor under the control of the controller 500 and a motor 420 for moving the camera 300 by the motor driver 410.

The operation of the target tracking apparatus of the image phone according to the present invention constructed as above will be described.

First, the A / D converter 610 is turned on according to the control of the control unit 500, A / D-converts the NTSC signal coming from the camera 300, and outputs it as an 8-bit digital signal.

At this time, when the tracking mode is turned on to start the telephone conversation, the original image coming from the camera 300 is A / D converted by the A / D converter 610 and stored in the current image storage unit 620.

The initial image stored in the current image storage unit 620 is input to the window setting unit 710 to determine the shape of the roof top plate area in which the initial window composed of 3 x 3 pixels is set in the portion where the target caret is located.

The brightness value of each pixel is searched by selecting pixels adjacent to the initial window in which the pixel value (N) of brightness values of the outermost pixels in the set initial window is calculated by the above equation (1). That is, the absolute value of the brightness value of the selected pixel is subtracted from the absolute value of the brightness value of the selected pixel, and the absolute value is compared with a threshold value that is greater than the threshold value. At this time, if the value obtained by subtracting the average value of the brightness values from the absolute value of the brightness value of the selected pixel as a result of the beg is not larger than the set threshold value, '1' is set to the selected pixel.

In this case, the above process is repeated until '1' is not set for all pixels adjacent to the set window.

The portion set to '1' is a portion of the brightness value in the initial position that is uniformly distributed, and is used in the shape of the upper plate region.

In accordance with the shape of the initial window NasK determined in this way, the top view is calculated with respect to the search area among the currently input images. At this time, it is assumed that the position of the initial talker is allot.

That is, the shifter 720 moves the window (x, y) of the previous image to the search area of the current image so that the upper plate can also calculate the upper plate degree by the calculation unit 730.

That is, the shifter 720 moves and stores the window of the previous image to calculate the NAE of the search area in the current image and the window of the previous image in the previous image storage 630.

The search area of the top plate area input from the shifter 720 and the search area of the current image input from the current image storing part 620 are calculated through the difference in the correlation calculation part 730 and the NAE is calculated. When the MAE is the smallest, 720) is the portion where the current talker is located.

The search area must be larger than the size of the initial window. In the present invention, an area including about 4 times the size of the rectangle including the initial window rate is used as the search area. That is, the search area has a size of 6 x 6 pixels with an initial window consisting of 3 x 3 pixels.

Since the calculation of the top plate 730 of the top plate also shows that the top view is the highest, that is, the search area having the smallest NAE, it is calculated by the difference between the previous position and the current position. The motion estimation unit 740 estimates motion using the motion information thus calculated.

Finally, the control unit 740 drives the motor driver 410 in response to the e-mail information inputted from the e-mail position estimation unit 740.

Accordingly, the camera 300 moves and a new image is input to the A / D converter 610. At this time, the controller 500 turns the A / D converter 610 on.

That is, in the controller 500, the A / D converter 610 and the current motion estimator 900 which turns on the current image storage unit 620 estimate the position of the robot to be in the dark position, 610 and the current image storage unit 620 to prevent the NTSC signal from being input from the camera 300. [

Through such an operation, the camera 300 is moved according to the position of the target, so that the camera 300 can track the target.

As described above, according to the present invention, the method and apparatus for tracking a target object of an image phone determine the shape of an effective window through an initial image processing process instead of a window of a conventional square hinge, It has the effect of improving the performance while enjoying the calculation amount.

Claims (9)

(100, 101, 102, 103, 104, 105) for determining the shape of the upper plate area by comparing brightness values of an initial window and neighboring pixels adjacent to each other in an initial image input in a tracking mode, A top plate 106, 107 for calculating a correlation between a previous image and a current image input according to the determined shape of the upper plate region; Estimating and driving the camera by estimating that the camera is a dark person according to the calculated upper plan view, and driving a motor (108, 109); And a call termination step (110,111) for repeating the calculation steps (106,107) and the town office estimation and the motor driving step (108,109) for the image input until the call is terminated. Target tracking method. The method according to claim 1, wherein the initial image is a video taken by a camera when the target is at a preset position. 3. The method of claim 1 or 2, wherein the determining of the correlation area shape comprises an initial image storage step (100,101) of converting an initial image input in a tracking mode into an A / D (Analog / Digital) (102) for setting an initial window at a position of a target window, an initial window setting step (102) for setting an initial window at a position of the target window, And determining a shape of the upper plate region according to the set ridge value, wherein the step (105) includes the steps of: (a) How to trace. 4. The method of claim 3, wherein the initial window comprises 3 x 3 pixels. 4. The method of claim 3, wherein the window waking step (103,104) comprises: calculating a pincushion value of brightness values of two outermost pixels of the set window; calculating a brightness value of pixels adjacent to the set window, (202,203,204,205) for calculating a difference value rate of a homogeneous value and comparing the threshold value with a set threshold value, and if it is determined that the pixel with the set value is adjacent to the set window, the process proceeds to the shape determination step (208) And a window setting step (206,207) for setting the average value calculation step (201) to the neighboring pixels of each pixel of the set window if there is a hag. 2. The method according to claim 1, wherein the top plate function for calculating the top plate map is a NA (Nean Absolute Error). An image storage means 600 for A / D-converting the input image in the tracking mode; A window setting unit 710 for determining the shape of the upper plate area by subtracting the brightness value of the initial image stored in the image storage unit 600 from the neighboring pixels adjacent to the window in accordance with the shape of the window set by the window setting unit 710; A top plate calculation means 730 for calculating a top plate degree by extracting a correlation region and a search region from a previous image and a current image output from the image storage means 600, The top plate estimating means 740 estimates the motion according to the top plan calculated by the top calculating means 730 and the operation of the image storing means 600. The estimating means 740 estimates And a control unit (500) for controlling the camera to move in accordance with the state of the town hall. The image phone according to claim 7, further comprising a shifter (720) for shifting the previous image stored in the image storage means (630) and outputting the upper panel to the calculation means (730) Tracking device. The image processing apparatus according to claim 7 or 8, wherein the image storage means (600) comprises an A / D converter (610) for A / D converting input image data in a tracking mode, A current image storage unit 620 for storing a video signal and a previous image storage unit 630 for storing a video signal output from the current image storage unit 620. [ Tracking device.