KR100365958B1 - A target trace controlling method for remote control camera with zooming function - Google Patents

Abstract

The present invention relates to a target tracking control method for a remote camera having a zoom function for automatically controlling a target to be viewed so as not to go out of a screen when a camera such as a video conference system or a surveillance camera is remotely zoomed .

The present invention determines the focus of expansion of a camera in that extensions of optical flows obtained through a zoom-in test are encountered before operating a camera, and when a user uses a cursor If the target is specified, the magnitude and direction of the optical flow of the obtained image are analyzed to adjust the parameter adaptively to suit the degree of camera zoom, thereby controlling the pan / tilt device so that the target is located at the center of the expansion, The zoom-in operation is performed. In this case, the position of the target is estimated based on the movement of the camera calculated by the optical flow, and the position of the target is accurately tracked by the image matching method to recognize the distance between the center of the expansion and the target, When the distance exceeds the pixel distance, the target is tracked and controlled by repeating the adaptive control of the pan / tilt device until a sufficient enlarged image is obtained for the target. Accordingly, the present invention enables quick zooming in a remote camera having a zoom function without any deviation of the target from the screen by a simple operation regardless of the initial zoom state.

Description

[0001] The present invention relates to a remote control camera with a zoom function,

The present invention relates to a method for controlling a target tracking of a remote camera having a zoom function for automatically tracking a target and displaying an enlarged image only by specifying a target with a cursor when the target is magnified and viewed using a remote camera.

Conventionally, in a camera control system for observing a position of interest, when a position to be watched by a user is designated by a cursor, a rotation angle of a pan / tilt device proportional to a pixel distance from a designated position to a center of the image And the camera was rotated. The panning / tilting device is a device that pans and rotates up and down and tilts. When the camera is placed on the panning / tilting device, a desired position in the up, down, left, and right directions can be seen .

However, if the zoom factor of the camera is changed, the characteristic of the camera control system changes. Therefore, when the control coefficients used in the past are used as they are, the target is often out of the screen because the camera is not properly controlled. For example, if you want to position a target that is 50 pixels horizontally away from the center of the screen at the center of the screen, if you could rotate the fan unit by 3 degrees and position it in the center in the zoom-out state, There is a case in which the target is moved out of the screen frequently due to the movement of much more pixels than the 50 pixels by rotating 3 degrees.

In other words, even for a control command of the same size, the degree of movement of the screen changes greatly according to the degree of zooming, so that the desired target moves much less than the user intends and feels frustrated, Occurs.

Therefore, in the conventional method, if the target is to be zoomed in, it is very inconvenient to use the zoom-out method, and then the target is located at the center of the screen and then zoomed in.

The present invention is conceived to solve the problems of the conventional art as described above, and it is an object of the present invention to adaptively control a pan / tilt device irrespective of a zoom state of a camera, And to provide a target tracking control method that can automatically locate a target at a center and confirm a target by zooming in.

1 is a conceptual diagram of a target tracking control method of a remote camera having a zoom function according to the present invention.

Figure 2 is an example of an optical flow when zooming in on a camera.

Figure 3 is an example of extending the optical flow of Figure 2 to locate the center of expansion;

FIGS. 4A to 4C are experimental examples of zooming in on the basis of the center of the image, in which the position of the target deviates from the center of the screen according to the zoom in

Figs. 5A to 5C are examples of zoom-in experiments performed on the basis of the focus of expansion (FOE). Figs. 5A to 5C show that the target does not move on the screen even when the zoom-

6 is a diagram showing an error change by the fixed coefficient control;

Figs. 7A and 7B are diagrams showing changes in errors and coefficient values according to zoom-out and half-zoom in, respectively,

According to an aspect of the present invention, there is provided a target tracking control method for a simple operation of a remote camera having a zoom function, comprising the steps of: extracting optical flows obtained through a zoom- When the user designates a target using a cursor, the center of the camera expansion is determined as a mutual meeting point. By analyzing the magnitude and direction of the optical flows obtained while controlling the pan / tilt device, the parameter is adaptively adjusted The target is located at the center of the expansion, and when it is determined that the target is sufficiently close to the center of the expansion, zoom-in is performed.

In addition, in performing zoom in, the position of the target is estimated based on the movement of the camera, and the position of the target is accurately tracked by the image matching method to recognize the distance between the center of the expansion and the target to determine whether the target deviates from the center of the expansion And repeats the process until the sufficient enlarged image is obtained with respect to the target using the adaptive control of the pan / tilt device when the distance from the center point is more than a certain distance from the center point.

The target tracking control method of a remote camera having a zoom function of the present invention is characterized in that the target tracking control method is recorded on a recording medium in the form of a program so that the tracking can be executed by a computer.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, preferred embodiments of a target tracking control method for a remote camera having a zoom function according to the present invention will be described in detail with reference to the accompanying drawings.

FIG. 1 is a conceptual diagram of a target tracking control method for a remote camera having a zoom function according to the present invention, which shows a flow chart on the left side and an effect of the camera image on the right side.

First, before the camera is operated in the video conference system, the surveillance camera, etc., the center of expansion is calculated through a zoom-in test (S101). Since the center of the expansion differs from camera to camera, once you have calculated the center of the expansion for a certain camera, you can continue to use that point as the center of the expansion unless you change the camera.

Here, the center of the extension is an extended trace of the optical flows appearing at the time of the zoom-in test, and it is decided that the points meet with each other. The optical flow can be defined as the distribution of the velocity vector in the image plane by the movement of the camera or the motion of the object (here, only the motion of the camera is fixed because the object is fixed). In the continuous image, (Indicated by a black line on the image in the image (a) of Fig. 1). Each time the camera is zoomed in, the resulting optical flows will appear differently, but the center of the expansion calculated using this will be similar. Therefore, in the embodiment of the present invention, the camera is zoomed in many times and the center of the expansion is obtained, and the average value of these is used as the center of expansion of the camera.

Referring to FIG. 1, when the optical flows in the image (a) shown on the right side of FIG. 1 are as shown in FIG. 2, the optical flows are extended as shown in FIG. It can be seen that the center of expansion of the laboratory camera 320 * 240 thus obtained is located in the lower left direction with respect to the centers 160 and 120 of the image as 162 and 128.

If the user specifies a target to be viewed as in the image b using the cursor in the state where the center of the expansion is obtained (S102), the target is moved to the center of the expansion as the image c (S103) . An adaptive parametric variable control technique is used to analyze the magnitude and direction of the optical flows obtained during the control of the pan / tilt device, and to position the target at the center of the expansion while adapting the parameters appropriately to the camera zoom state.

The adaptive parameter variable control technique is a technique for determining the appropriateness of the control by analyzing the optical flow obtained during the control of the pan / tilt device and adaptively selecting the appropriate parameter. The PD (Proportional-plus-Derivative ) Based on the control technique, it is repeatedly controlled until the position of the target is located at the center of the expansion.

In detail, in order to center the cross-shaped clicked position in the image (b) of FIG. 1, the camera should be appropriately panned to the left and tilted upward by using a pan / tilt device. At this time, the optical flow is analyzed to determine where the target has moved. If the target has been moved to the center point of the expansion, then the control is appropriate; if it is less, it moves further; if it is more, it should correct the parameter value according to Equation 2 and then to the center point of the expansion.

Therefore, it is necessary to determine the appropriate coefficient for precise control. The relationship between the control input calculated using the optical flow and the control effect is analyzed and determined. In this case, And differential coefficient Lt; The control method for

First, a constant control signal of a sufficiently small size is applied, and as a result, the moved pixel distance is measured, and the initial proportional coefficient ( ).

In addition, since an initially determined coefficient value may become an inappropriate value due to oscillation during control depending on the degree of zooming, if the sign of the error is changed based on the center point of the image during the control, .

here, Is the error that you tried to correct between the center of the target and the expansion, Is an error after actually being corrected by application of a control signal.

By experiment And the control signal to be applied is calculated as shown in Equation 3. " (3) "

Then, if it is determined that the target is sufficiently close to the center of the expansion, a zoom-in (or zoom-out) is performed (S104) and it is determined whether the target deviates from the center of expansion by more than a certain standard (S105). That is, in performing the zoom-in (or zoom-out), the position of the target is estimated based on the movement of the camera, and the distance between the center of the expansion and the target is recognized by the image matching method to determine whether the target deviates from the center of the expansion Monitor.

Here, a technique for recognizing the target position to be used will be described. The target position recognition technology is a technique for acquiring accurate camera motion information from an image and tracking the target without missing. In the embodiment of the present invention, stable feature points continuously detected in an adjacent frame are extracted for accurate matching. Since it takes a long time to obtain the optical flow for each value of the image, it extracts feature points that can represent the image. If the feature points are continuously detected in the adjacent frames of successive images, the movement position of the feature points can be accurately obtained Therefore, stable feature points are extracted.

After extracting the stable feature points, matching pairs are weighted using weighted correlation according to the distance in the comparison area, and the positions calculated by the matching position and the camera movement are compared to eliminate the false matching pair Make sure you get the correct optical flow. The camera movement is calculated using the optical flow distributed over the entire region of the image obtained in this manner, the movement position of the target is estimated using the calculated camera movement, and the weighted value is calculated for each pixel in the predetermined region based on the estimated position To accurately track the location of the target.

As a result of the determination in step S105, if the target deviates from the center of the expansion by more than a certain pixel distance, the process proceeds to step S103 to move the target to the center of the expansion. Otherwise, a sufficient zoom state, that is, (S106). That is, if the user inputs a desired number of times of zooming in advance, the user performs zooming until the zooming is performed, and it is determined whether or not zooming in has been performed the number of times.

As a result of the determination in step S106, if the zoom state is not sufficient, the above-described steps S104 to S106 are repeated until a sufficient zoom state is obtained.

The center of the expansion and the center of the image are set as the center points, respectively.

FIGS. 4A to 4C are experimental examples of camera zooming performed on the basis of the center of an image. FIGS. 5A to 5C are examples of zooming in experiments based on the center of expansion according to an embodiment of the present invention. will be.

As shown in the drawing, when the center of the image is referred to, the initial center position (the power display portion on the uppermost one of the three LEDs of the target) is moved in the upper right direction (see Figs. 4A to 4C). On the other hand, in the case where the center of expansion is used as a reference, it can be seen that the initial center position does not move but remains in the center even after zooming (refer to FIGS. 5A to 5C)

In zooming in, we compare and compare the effect of target tracking control by using fixed coefficient and variable coefficient.

FIG. 6 shows an error change by the fixed coefficient control, and FIGS. 7A and 7B show changes of errors and coefficient values according to zoom-out and half-zoom in, respectively, in the variable coefficient control.

In order to keep track of the target, the fixed coefficient determined in the zoom out state is set to a value smaller than a value to be corrected by zooming in the camera during the control for positioning the target in the center of the screen, When the large value is input and the error is repeated, the target will be out of the screen at the end of the graph. On the other hand, in the case of the embodiment of the present invention in which the initial zoom magnitude is considered and the coefficient that varies when the camera zooms in is used, the zoom magnification of the camera is increased as shown in the upper diagram of FIG. 7A (initial zoom- Regardless, you can see that the errors are getting smaller. 7A and 7B also show control coefficients adaptively changing during control ), And when the sign of the error is changed, it is corrected to an appropriate coefficient value and controlled.

The features of the present invention described above are as follows.

1. When zooming in to observe a target object, the target is positioned at the center of the expansion, and then the zoom-in is performed. Therefore, the target does not move on the screen during zooming. Therefore, since the control for moving the target is not repeatedly required, it is possible to quickly obtain the near enlarged image of the target.

2. When there is a certain amount of zoom in, there is no inconvenience to zoom in and out of an arbitrary image position, as in the conventional method, and in the present invention, After zooming in is possible.

3. Use optical flow to measure the control effect.

4. You can obtain enlarged close-up images by specifying the position of interest by the cursor on the screen.

These features of the present invention are tabulated in comparison with the existing methods.

By item Conventional method The method of the present invention Target movement reference point for zoom Center of the image screen: Objects in the center of the image move to the edge during zooming. Center of expansion: Objects in the center of the zoom do not move during zooming How to verify proximity of a given location Zoom out to move the target to the center of the screen, After moving the target to the center of the expansion, Control method Open loop method Disadvantages: Do not compensate for errors Closed-loop method using optical flow as control information, eliminating errors User's instructions Indication by cursor Indication by cursor

The present invention as described above is recorded on a computer-readable recording medium and can be processed by a computer.

As described above, the automatic target tracking control method for a remote camera having a zoom function of the present invention is a method for controlling a camera such as a video conference system or a surveillance camera remotely by zooming in on a target to be viewed, The system automatically controls the panning / tilting device by tracking the target to position the target at the center of the expansion of the image screen and displays the enlarged image through the zooming in.

Accordingly, the present invention has an effect that a remote camera having a zoom function can zoom in quickly without moving the target through a simple operation without zooming out before zooming in, irrespective of an initial zoom state.

Although the preferred embodiments of the present invention have been disclosed for illustrative purposes, those skilled in the art will appreciate that various modifications, additions and substitutions are possible, without departing from the scope and spirit of the invention as disclosed in the accompanying claims. In addition, it is a matter of course that various modifications and variations are possible without departing from the scope of the technical idea of the present invention by anyone having ordinary skill in the art.

Claims (6)

A target tracking control method for zooming remotely from a camera such as a video conference system or a surveillance camera, A first step of extending the optical flow through the zoom-in test before the camera is operated, and determining the point of mutual encounter as the center of the camera expansion; A second step of controlling the pan / tilt device such that a target designated by a user's cursor operation is moved to the center of the expansion; When the target is located at the center of the expansion, zoom-in is performed. When zooming in, the distance between the center of the expansion and the target is recognized by the image matching method to monitor whether the target deviates by more than a predetermined distance from the center of the expansion A third step; And And a fourth step of performing the second step if the target is out of a predetermined reference at a center of the expansion in the third step and stopping the zooming at a time when a sufficient enlarged image for the target is obtained, In the second step, the control of the pan / tilt apparatus is performed by analyzing the size and direction of the optical flows obtained while controlling the pan / tilt apparatus, and as a result of the analysis, a parameter for controlling the movement of the current pan / And controlling the operation of the pan / tilt device with a variable parameter. delete delete The method according to claim 1, The adaptive control for the pan / tilt device performed in the second step includes a control signal applied according to the target designation and an initial proportional coefficient KP as a pixel distance moved by the target to the center of expansion by the control signal, ; Determining If the sign of the error changes on the basis of the center point of the image during the movement according to the control signal, an error (errori-1) to be corrected between the target and the center of the expansion and an error ) Into the following equation (1) to update the proportional coefficient KP, And a new control signal for controlling the movement of the pan / tilt apparatus is determined by substituting the differential coefficient (KD) determined as an appropriate proportional value of the proportional coefficient KP into the following equation (2) Wherein the target tracking control method comprises the steps of: (1) (2) delete A computer-readable recording medium recording a target tracking control program for zooming remotely from a camera such as a video conference system or a surveillance camera, A first step of extending the optical flow through the zoom-in test before the camera is operated, and determining the point of mutual encounter as the center of the camera expansion; Controlling a pan / tilt device such that a target designated by a cursor operation of a user is moved to the center of the extension, wherein the control of the pan / tilt device is based on the magnitude and direction of the optical flows obtained during the control of the pan / A second step of varying a parameter for controlling the movement of the current pan / tilt device as a result of analyzing and controlling the operation of the pan / tilt device with a variable parameter; When the target is located at the center of the expansion, zoom-in is performed. When zooming in, the distance between the center of the expansion and the target is recognized by the image matching method to monitor whether the target deviates by more than a predetermined distance from the center of the expansion A third step; And And a fourth step of performing the second step if the target is out of a predetermined reference at the center of the expansion in the third step and stopping the zooming at a time when a sufficient enlarged image of the target is obtained if the target does not deviate, A computer-readable recording medium on which a program for enabling tracking of a target with a camera is recorded.