KR101456340B1 - Camera Movement Control Apparatus and Method using Motion Vector - Google Patents

Abstract

A camera movement control apparatus and method thereof are disclosed. A motion control apparatus for a camera according to the present invention includes: a reference image selection unit for selecting a reference image from an input image signal; A difference operation performing unit for performing a difference operation on the current image based on the reference image selected by the reference image selecting unit; A motion vector calculation unit for estimating a motion of an object based on a current image and a reference image and calculating a predicted motion vector when the result of the difference operation performed by the difference operation performing unit is greater than a preset threshold value; And a moving direction control unit for controlling the moving direction of the camera based on the predicted motion vector calculated by the motion vector calculation unit.

Description

[0001] The present invention relates to a camera movement control apparatus,

The present invention relates to an apparatus and method for controlling movement of a camera, and more particularly, to a motion control apparatus and method for controlling movement of an object in a video signal inputted through a camera, And more particularly, to a camera movement control apparatus capable of actively monitoring an object by continuous monitoring.

In general, a camera for capturing an image is used to analyze the motion of an object based on a frame of the input image at a predetermined time interval, and to monitor movement of the object based on the motion of the analyzed object.

However, since such a surveillance camera captures an image in a state in which there is no background change, there is a problem that continuous monitoring can not be performed if the moving object deviates from the surveillance range of the camera.

Also, even if the camera moves according to the movement of the object, it is only a level that the manager controls the direction of the camera by adjusting the joystick and the like.

Disclosure of Invention Technical Problem [8] Accordingly, the present invention has been made keeping in mind the above problems occurring in the prior art, and it is an object of the present invention to provide an image processing apparatus, And a method of controlling the movement of the camera.

According to an aspect of the present invention, there is provided an apparatus for controlling movement of a camera, comprising: a reference image selector for selecting a reference image from an input image signal; A difference calculation unit for performing a difference operation on the current image based on the reference image selected by the reference image selection unit; A motion vector calculator for estimating a motion of an object based on a current image and a reference image and calculating a predicted motion vector when the result of the difference computation performed by the difference computation unit is greater than or equal to a preset threshold value; And a moving direction control unit for controlling the moving direction of the camera based on the predicted motion vector calculated by the motion vector calculating unit.

The difference operation performing unit may divide an image in a screen into search areas having an arbitrary size, and generate motion vectors for the respective search areas to perform a difference operation.

The difference operation performing unit may perform a difference operation on the next image if the result of the difference operation on the current image is smaller than the threshold value.

According to an aspect of the present invention, there is provided a method of controlling movement of a camera, the method comprising: selecting a reference image from an input video signal; Performing a difference operation on the current image based on the reference image selected by the reference image selection step; Calculating a predicted motion vector by estimating a motion of an object based on the corresponding image and the reference image if the result of the difference operation performed by the difference operation performing step is greater than or equal to a set threshold value; And controlling the moving direction of the camera based on the predicted motion vector calculated by the motion vector calculating step.

The difference calculation step may divide the image in the screen into search areas having an arbitrary size, and generate motion vectors for the respective search areas to perform a difference operation.

In the performing the difference operation, if the result of the difference operation on the current image is smaller than the threshold value, the difference operation can be performed on the next image.

According to the present invention, it is possible to perform a fixed monitoring of an object by detecting a motion of an input video signal to predict a motion vector of the object, and controlling the overall moving direction of the camera based on the predicted motion vector of the object However, active monitoring can be performed according to the motion of the object.

1 is a schematic view of a camera movement control apparatus according to an embodiment of the present invention.
2 is a diagram showing an example of a block matching method.
3 is a diagram showing a relationship among frames for calculation of a motion vector.
FIG. 4 is a diagram showing an example of setting an accumulated occupancy vector value between frames and an object area.
5 is a diagram illustrating a range of a search area.
6 is a flowchart illustrating a camera movement control method according to an embodiment of the present invention.
7 is a diagram showing an example of a prediction image through simulation.
FIG. 8 is a diagram illustrating motion vectors and calculation time results through global motion estimation at intervals of 10 frames.
FIG. 9 is a diagram showing motion vectors and arithmetic time results using 9 search areas at intervals of 10 frames.
FIG. 10 is a diagram illustrating motion vectors and calculation time results through global motion estimation at intervals of 40 frames.
FIG. 11 is a diagram showing motion vectors and calculation time results using 9 search areas at intervals of 40 frames.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the following description, a detailed description of known techniques well known to those skilled in the art may be omitted.

In describing the constituent elements of the present invention, the same reference numerals may be given to constituent elements having the same name, and the same reference numerals may be given thereto even though they are different from each other. However, even in such a case, it does not mean that the corresponding component has different functions according to the embodiment, or does not mean that the different components have the same function. It should be judged based on the description of each component in the example.

In the following description of the embodiments of the present invention, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear.

In describing the components of the present invention, terms such as first, second, A, B, (a), and (b) may be used. These terms are intended to distinguish the constituent elements from other constituent elements, and the terms do not limit the nature, order or order of the constituent elements. When a component is described as being "connected", "coupled", or "connected" to another component, the component may be directly connected or connected to the other component, Quot; may be "connected," "coupled," or "connected. &Quot;

1 is a schematic view of a camera movement control apparatus according to an embodiment of the present invention.

Referring to FIG. 1, a camera movement control apparatus 100 according to an exemplary embodiment of the present invention includes a reference image selection unit 110, a difference calculation unit 120, a motion vector calculation unit 130, (140).

The reference image selection unit 110 selects a reference image from a video signal input through a camera (not shown).

The difference operation performing unit 120 performs a difference operation on the current image based on the reference image selected by the reference image selector 110. [ At this time, the difference operation performing unit 120 may perform a difference operation using an intra prediction method, an inter prediction method, or the like of the video signal encoding method. Such a difference operation performing method is well known in the art, and a detailed description thereof will be omitted here.

The motion vector calculator 130 estimates the motion of the object based on the current image and the reference image and calculates a predicted motion vector if the result of the difference computation performed by the difference computation unit 120 is equal to or greater than a preset threshold value . That is, the motion vector calculation unit 130 estimates the motion of the object only when the result of the difference operation of the current image with respect to the reference image is equal to or greater than the threshold value.

The movement direction control unit 140 controls the moving direction of the camera based on the predicted motion vector calculated by the motion vector calculation unit 130. [ That is, when the motion vector calculating unit 130 calculates a predicted motion vector for an object, the velocity and direction of the object can be predicted based on the calculated predicted motion vector, The direction of the camera can be controlled.

Conventional techniques related to moving object tracking technology can be divided into a method using a difference operation and a method using an optical vector.

The method of pixel difference calculation is generally aimed at monitoring in a situation where there is no change of the background in the form of a fixed camera capturing a screen. Recently, there are various methods of detecting the area of the moving object by using the video information, and the background difference, the feature of color or shape, and the method of using motion information are representative.

The method of using the background difference is to combine the background images obtained from different angles into one mosaic image before detecting the motion information. After that, the background image of the current image is found in the mosaic image and compared with the current image to detect the motion area.

 In the method of using the color and shape information of an object, a moving object is modeled using a color or shape feature value, and a region that best matches the model is determined as a region of the moving object. Another method of using motion information is to use a difference operation to define a region having a large difference value according to time of a pixel constituting two consecutive frames as a motion. Global Motion Estimation (GME) is performed to obtain the ordinary difference operation, but there is a problem that the calculation amount is large.

Motion detection and tracking generally use a computation-based algorithm, which is largely divided into global motion estimation (GME), global motion compensation (GMC), and localization of moving image ranges Localization. This is a method of estimating the motion between two images caused by camera motion by dividing the image into the previous image current image with a certain frame. However, this method has the following problems.

First, it is impossible to detect the exact moving object area according to the speed of the moving object shown in the image. There is no problem in obtaining a difference operation using a continuous image having a constant time interval. However, in the case of an object moving at a high speed, the detection result may include a part of the object as well as a background. On the other hand, when the object moves too low, there is a problem that only a part of the object is detected. Therefore, it is necessary to select a previous image corresponding to the speed of the moving object.

Second, GME takes a lot of time because it is computationally expensive. Generally, since the previous image and the current image are fully searched using all the pixels in the image, the calculation amount is increased

In order to obtain an inter-frame motion vector (MV), the subject of the image must be grasped first, and this refers to the object that becomes the center of motion in the image. Basically, there are cases where the background is centered and the object is centered. There are pixel iteration and block matching methods for predicting MVs. The pixel iterative method for predicting a motion vector on a pixel-by-pixel basis requires no search area determination, predicts a more accurate value, can detect local motion, and is effective for less motion.

The block matching method assumes that all pixels in a block have the same motion and that the motion of the object is a parallel motion. Each block is independent and this method is robust to noise and low computational complexity and is useful for real-time processing. Generally, blocks are 8x8 or 16x16. The smaller the size of the block, the more accurate the motion vector can be predicted, but the larger the amount of computation.

As shown in FIG. 2, the block matching method determines a search position indicating a macroblock (MB), obtains a sum of absolute difference (SAD) thereof, and determines a direction having the smallest value as a motion vector.

Conventional techniques related to object tracking techniques are representative of using a difference operation as described above. The method of pixel difference calculation is generally aimed at monitoring in a situation where there is no change of the background in the form of a fixed camera capturing a screen.

As a basis for determining the motion in the difference operation, as shown in FIG. 3, if there is a difference between the pixel values of the corresponding lattice points in the previous frame and the current frame, it can be determined that there is some motion. Therefore, a difference operation between corresponding pixels at the same position is performed between two frames. This difference operation can be expressed as Equation (1). x and y denote the positions of the intersections on the grid structure of each frame, F _t denotes the current frame, F _{t -1} denotes the previous frame, and G () denotes each pixel value.

[Equation 1]

However, it is not meaningful to obtain the difference image corresponding to the same coordinates in the situation of changing the direction of the equipment to be controlled according to the tracking of the motion. As shown in FIG. 3, the background of the two adjacent frames is different due to the movement of the camera. That is, the pixel at the specific position in the previous frame is located at the position shifted by the movement variable AP and DELTA T of the camera in the new frame. For this reason, the target area for obtaining the difference of the pixel values is limited to an intersecting area between two frame areas. This is because the non-intersecting area between adjacent frames taken at different times is relatively narrow compared with the entire area of the video frame, so that it is possible to detect the motion by only the pixel difference in the intersecting area. Therefore, Δi and Δj according to the movement variables ΔP and ΔT of the camera should be considered so that comparison can be made for the same pixel in two frames.

Since the images of each frame are influenced by the surrounding environment such as illuminance and focus, the pixel value at the corresponding position between two frames taken at the adjacent time It is not the same. Therefore, a threshold value is set so as to accommodate the variability according to such an environment. When the difference calculation result of the position is less than the threshold value, the pixel value is regarded as the same pixel value.

FIG. 4 is a diagram illustrating an example of setting an accumulated motion vector value and an object region between frames.

The attributes of the moving object can be the size of the object, the center of gravity, and the motion vector of the object. If Equation (2) is expressed, if the cumulative value of the MV accumulated by the area contribution is greater than 9 pixels, the neighboring block has the shape of the object overlapping more than half a circle. do. In the case of more than 12 pixels, it is considered that the object is almost exited from the corresponding block, and the corresponding block is removed as the background area instead of the object area, and a new object area is set by finding the most similar part using BMA.

&Quot; (2) "

In order to detect a motion vector in the screen, the subject of the image must first be grasped. In most of the images, the subject of the motion base control is mostly used as the background. The reason for this is that there are many scenes in the third person centered around the object and there are many scenes where the background moves around the object when the object moves. Therefore, motion-based control is often performed according to background motion.

When an object becomes a subject of an image, the characteristic is generally that there is little background motion and the object moves or moves. On the basis of this, the reference image selector 110 can select a start image having a motion vector equal to or greater than a set value as a reference image. However, the method of selecting the reference image by the reference image selection unit 110 is not limited to the described method, and various methods can be used.

The difference operation performing unit 120 may calculate the motion vector of the entire image using the BMA in order to determine the characteristics of the image in the screen. Next, the motion vector of the block unit is divided into nine local motion vectors (LMVs) having an arbitrary size as shown in FIG. 5 according to the size of the image, and the values of the nine LMVs The characteristics of the image can be grasped.

First, 9 search areas are specified, and motion vectors are generated in units of 16 × 16 blocks. At this time, the range of the 9 search areas can be arbitrarily changed.

6 is a flowchart illustrating a camera movement control method according to an embodiment of the present invention.

1 to 6, the reference image selector 110 selects a reference image from an input image signal (S110).

The difference operation performing unit 120 performs a difference operation on the current image with reference to the reference image selected by the reference image selecting unit 110 (S120). At this time, the difference operation performing unit 120 may divide the image in the screen into search regions having arbitrary sizes, and generate motion vectors for the respective search regions to perform difference operations. The difference operation performing unit 120 may perform a difference operation on the next image if the result of the difference operation on the current image is smaller than the threshold value.

The motion vector calculator 130 estimates the motion of the object based on the current image and the reference image and calculates a predicted motion vector if the result of the difference computation performed by the difference computation unit 120 is equal to or greater than a preset threshold value (S130). That is, the motion vector calculating unit 130 predicts a motion vector and predicts the next image when the moving speed of the object is faster than the predetermined value.

The movement direction control unit 140 controls the moving direction of the camera based on the predicted motion vector calculated by the motion vector calculation unit 130 (S140).

In the embodiment of the present invention, GMV (Global Motion Vector) generation and extraction are performed using the block matching method. The above method was experimented by using Visual Studio 10 software. In the experiment, pre-recorded CIF format image data was used. Experiments were carried out by excluding video images whose images were suddenly converted. The threshold value for the difference operation was set to 30, and the experiment was conducted. If the difference is lower than the threshold, the motion of the background and the object is considered to be small, but the motion is continuously analyzed. A method of determining the direction of motion according to the MV obtained in the motion estimation process is as follows.

That is, the moving direction information is used only when any one of the values of x and y is equal to or greater than the threshold value of 2 or -2. If both of the values are less than the threshold value, the image analysis is performed again without using the moving direction information. First, the first frame and the second frame are input, and motion estimation (ME) between the two frames is performed. Then, a prediction frame (Prediction Frame) and an error image (Error Image) are obtained through the obtained MV, and the moving directions of the up, down, left, and right are obtained as an average of MV.

The frames used in the simulation of Fig. 7 are Tabletennis and Harry Potter images. The MV is obtained between the first and second frames, and the third prediction image is obtained. After the prediction image thus obtained is registered as the next reference image, the same process is performed using the following frame. First, the MV obtained by calculating the entire range of the image and the calculation time in the experiment for obtaining the MV obtained when the frame interval in the image is 10 is shown in FIG.

If the frame interval in the image is short, the movement of the background and the object is small. Therefore, it is determined that there is no motion due to the threshold value in the difference calculation process. In case of Harry Potter image, the motion of the image is very fast. In this case, the variation of the background and the object is large, and it is determined that there is motion in the difference calculation process. The subsequent MV values were 3 and 0, and the direction of movement with respect to the right direction was obtained.

The result of FIG. 9 using the 9-Search Range is not MV computed when the motion of the background and the object is insignificant as in FIG. In the Harry Potter image with a large variation of the background and object, the MV operation after the difference operation is performed, and the MV direction is 2, 0, and the movement direction to the right direction is obtained.

The data in FIG. 10 is the MV result obtained by setting the frame interval of the image to 40. FIG. In the case of Tabletennis, the subject of the movement is a small table tennis ball. In the case of other images, the motion estimation process was performed as the frame interval increased, and the direction of movement for each image was obtained.

The data in Fig. 11 is the moving direction obtained by using the 9-Search Range in the same frame as Fig. The computation time is reduced compared to global motion estimation, and MV is similar to global motion estimation. The moving direction values of up, down, left, and right according to MV were also obtained.

In the embodiment of the present invention, MV extraction between frames of an image, a predicted image using the extracted MV, and moving direction data are obtained. When comparing the actual image with the predicted image, it was confirmed that almost similar motion direction and image can be obtained even with the naked eye. It is possible to monitor more actively based on the moving direction designation information, rather than monitoring the fixed position of the camera equipment using the moving direction data obtained by this experiment.

The present invention is not necessarily limited to these embodiments, as all the constituent elements constituting the embodiment of the present invention are described as being combined or operated in one operation. That is, within the scope of the present invention, all of the components may be selectively coupled to one or more of them. In addition, although all of the components may be implemented as one independent hardware, some or all of the components may be selectively combined to perform a part or all of the functions in one or a plurality of hardware. As shown in FIG. In addition, such a computer program may be stored in a computer-readable medium such as a USB memory, a CD disk, a flash memory, etc., and read and executed by a computer, thereby implementing embodiments of the present invention. As the storage medium of the computer program, a magnetic recording medium, an optical recording medium, a carrier wave medium, or the like may be included.

Furthermore, all terms including technical or scientific terms have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs, unless otherwise defined in the Detailed Description. Commonly used terms, such as predefined terms, should be interpreted to be consistent with the contextual meanings of the related art, and are not to be construed as ideal or overly formal, unless expressly defined to the contrary.

The foregoing description is merely illustrative of the technical idea of the present invention and various changes and modifications may be made by those skilled in the art without departing from the essential characteristics of the present invention. In addition, the embodiments disclosed in the present invention are not intended to limit the scope of the present invention but to limit the scope of the technical idea of the present invention. Accordingly, the scope of protection of the present invention should be construed according to the claims, and all technical ideas within the scope of equivalents should be interpreted as being included in the scope of the present invention.

Claims (6)

A reference image selection unit for selecting a reference image from an input image signal;
A difference calculation unit for performing a difference operation on the current image based on the reference image selected by the reference image selection unit;
A motion vector calculator for estimating a motion of an object based on a current image and a reference image and calculating a predicted motion vector when the result of the difference computation performed by the difference computation unit is greater than or equal to a preset threshold value; And
And a movement direction control unit for controlling a movement direction of the camera based on the predicted motion vector calculated by the motion vector calculation unit,
Wherein the difference operation performing unit comprises:
Wherein if the result of the difference operation is smaller than the threshold value, the difference operation is performed on the next image.
The method according to claim 1,
Wherein the difference operation performing unit comprises:
Wherein the motion of the camera is calculated by dividing the image in the screen into search regions having an arbitrary size and generating motion vectors for the respective search regions.
delete Selecting a reference image from an input video signal;
Performing a difference operation on the current image based on the reference image selected by the reference image selection step;
Estimating a motion of an object based on a current image and a reference image and calculating a predicted motion vector if the result of the difference operation performed by the difference operation performing step is greater than a preset threshold value; And
And controlling the moving direction of the camera based on the predicted motion vector calculated by the motion vector calculating step,
The step of performing the difference operation includes:
And performing a difference operation on the next image if the result of the difference operation on the current image is smaller than the threshold value.
5. The method of claim 4,
The step of performing the difference operation includes:
A moving image is divided into a search area having an arbitrary size and a motion vector is generated for each search area to perform a difference operation.
delete

Images