KR100443677B1 - Camera motion detecting method using motion vector quantization for moving picture - Google Patents

Abstract

In the motion vector quantization method of a video according to the present invention, a quantization section is divided into horizontal / vertical two-dimensional grids for each picture (frame) in a compressed video using a compression method to which a motion compensation method is applied. A histogram of the motion vectors is generated by detecting the number of motion vectors mapped to the quantization interval.

Here, in dividing the quantization section into a horizontal / vertical two-dimensional grid, the motion vector (0, 0) is positioned at the center of the quantization region, and the upper, lower, left, and right sides of the motion vector are positioned in the maximum range of the designated motion vector. Set the quantization section to the rectangular area.

In addition, the camera motion detection method according to the present invention comprises the steps of dividing a quantization interval into a horizontal / vertical two-dimensional lattice for each picture (frame) in the compressed video using the compression method to which the motion compensation method is applied; ; Generating a histogram for the motion vector; Detecting camera motion from a distribution of bin values of the motion vector histogram; It includes.

In detecting the camera motion from the distribution of the bin values of the motion vector histogram, it is determined that the camera motion is generated when the total frequency of the motion vector histogram is equal to or greater than a predetermined threshold for the total number of macroblocks.

Description

Camera motion detecting method using motion vector quantization for moving picture}

The present invention provides a method of quantizing a motion vector in order to efficiently extract motion information from a compressed video using a compression method using a motion compensation method such as MPEG-1 / 2, H.261, H.263, and the like. The present invention relates to a motion vector quantization method of a video capable of effectively detecting a motion picture and a camera motion detection method using the same.

In the shift mode of the conventional moving picture, the shift speed is fixed at the same speed constantly throughout the entire shift. However, such variable speed technology may not satisfy the viewer's visual characteristics when performing fast or slow playback.

For example, in the case of fast playback at constant speed, the scene changes rapidly so that the user cannot recognize the scene, and the scene changes slowly. By proceeding, there is no feeling that fast reproduction is performed.

In addition, in the case of slow playback, a scene that changes the screen over time is often passed quickly even though the user wants to recognize the scene slowly. On the contrary, in a slow scene, the screen is slower. This transition will make you feel boring.

However, if the fast view is usually performed for the purpose of high speed search, the slow view is performed for the purpose of looking at a specific scene more closely, and in view of the user's visual point of view, the slow view is somewhat slow in the place where the scene change is severe. You will expect the scene to change, and you will expect the scene to change somewhat faster where the change in the scene is slow.

However, there is a limit in the fixed constant fast / slow that can not meet the needs of the user.

Therefore, the above problem can be solved if the adaptive speed automatic adjustment of the moving image adaptively varying the degree of variable speed according to the image change rate in the variable speed section is performed.

In such a technique, by detecting a cut and using a video difference between the cut and the cut and an image difference between adjacent frames, the playback speed is varied within the execution section at a faster or slower rate depending on the rate of change of the image, depending on the dynamic characteristics of the image. A technique for adaptive speed automatic adjustment of a moving picture in which a speed in a shift playback section is adaptively changed to be suitable for human visual characteristics has been proposed (Korean Patent Laid-Open Publication No. 1999-0081317, November 15, 1999). ).

As described above, the concept of a system for automatically controlling the playback speed according to the complexity of the screen has been introduced as a display system for automatically controlling the playback speed based on the scene change rate. However, in this technology, the scene change is defined as a difference image, and is based on the 'screen complexity' based on the image difference between adjacent frames.

Therefore, when defining motion with a difference image, all pixels need to be compared, thereby limiting the processing time. Also, when the screen complexity is defined by the difference image, even though the motion (movement) of the object is the same, the object having a large color difference with the background is calculated as a large one even though the movement is small, so that the part is played slowly. Will not actually reach the desired playback speed. In other words, it is because the complexity of the screen itself is regarded as a reflection of the content complexity, and the screen complexity is calculated as a differential image, rather than the playback speed control centered on meaningful content development in order to actually understand the video.

In addition, in the actual content development of video, it is hard to see that the difference between front and rear image faithfully reflects the meaning of the content development, and overlook that the complexity of the content itself may actually be felt differently depending on the story development in addition to the simple image difference. have. In other words, for example, in the case of drama, the scenes in which characters are talking may continue tediously, but if the position of the camera changes frequently at this time, this part may be judged to be complicated because the difference in front and rear image is increased. There is a possibility of slow playback. However, even if the scene where the characters actually talk continues for a long time tediously, it may be better to play this part rather quickly.

This is because it is difficult to automatically calculate and judge the 'content complexity' that actually reflects the content according to the genre, use, and content of the video using only the difference image. From the point of view of the actual content development of the video, the 'content complexity' of the video is not only the difference image, but also motion information, shot information, face information of the character (specific object detection information that is the center of content development), text information, and audio information This is because it reflects the 'complexity of content' felt by real people when the back is considered comprehensively.

In addition, in the field of digital video indexing, camera or object motion information becomes meaningful indexing information. In the video quick view system using the indexing information (Korean Patent Application No. 2000-0057722, 2000.09.30), the fast view speed can be automatically adjusted by using the index information of how complex the motion of a specific section is. The camera may separately index zoomed-in, zoomed-out or tracked shots.

And, when camera motion is detected, fast forward speeds can be eliminated in shot change detection or in zoom-in scenes (assuming they are zoomed in as important scenes) in zoom-in scenes. You can also lower it so you can see more slowly.

However, when the camera motion is detected through this method, a large amount of computational time is used. In particular, when such a video indexing method is implemented on a home TV or a TV set-top box as a personal video recorder (PVR), implementation constraints occur in processing performance, memory usage, data bandwidth, and the like. Accordingly, there is a limitation in that the intelligent video indexing method is applied to a system having limited system performance such as a PVR.

The present invention has been made in consideration of the above-described conditions, and a quantization section is divided into horizontal / vertical two-dimensional lattice for each picture (frame) in a compressed video using a compression method to which a motion compensation method is applied. It is an object of the present invention to provide a motion vector quantization method of a video capable of generating a motion vector histogram containing direction and magnitude information of a motion vector by reducing the amount of data for the motion vector by generating the motion vector histogram.

In addition, the present invention provides a camera motion detection method that can quickly detect the camera motion from the distribution of the bin of the motion vector histogram through the analysis of the motion vector histogram generated using the motion vector quantization method of the video as described above. There is another purpose for that.

1 is a view for explaining a motion vector quantization and histogram of a video according to the present invention.

2 is a view showing an example of motion vector quantization interval division in motion vector quantization of a video according to the present invention;

3 is a diagram illustrating an example of motion vector quantization of a video according to the present invention;

4 is a diagram illustrating types of general camera motion.

5 is a diagram illustrating each macroblock motion vector and a motion vector histogram when 'track' or 'pan' is generated in a camera motion by a motion vector quantization method of a video according to the present invention.

6 is a diagram illustrating each macroblock motion vector and a motion vector histogram when 'boom' or 'tilt' is generated in a camera motion by a motion vector quantization method of a video according to the present invention.

7 is a diagram illustrating each macroblock motion vector and a motion vector histogram when 'dolly' or 'zoom' occurs in camera motion by a motion vector quantization method of a video according to the present invention.

In order to achieve the above object, the motion vector quantization method of a video according to the present invention comprises a horizontal / vertical two-dimensional lattice for each picture (frame) in a compressed video using a compression method to which a motion compensation method is applied. The characteristic is that the quantization interval is divided and a histogram of the motion vectors is generated by detecting the number of motion vectors mapped to each quantization interval.

Here, in dividing the quantization section into a horizontal / vertical two-dimensional grid, the motion vector (0, 0) is positioned at the center of the quantization region, and the upper, lower, left, and right sides of the motion vector are positioned in the maximum range of the designated motion vector. The characteristic is that the quantization section is set to the rectangular area.

Also, in setting the maximum range of the specified motion vector, a value above a predetermined value is set to the maximum range of the motion vector so as to include motion vectors that can appear in a general video, or in f_code of MPEG-1 / 2. The feature is that the maximum range of the motion vector is set by using the range of the motion vector limited as it is or by converting it linearly.

Further, in dividing the motion vector into horizontal / vertical two-dimensional grids, the motion vector space is equally divided, the motion vector space is linearly and unevenly divided, or each division point is designated. This feature is characterized by non-linear partitioning of motion vector space.

In addition, the camera motion detection method according to the present invention in order to achieve the above object,

Dividing a quantization section into a horizontal / vertical two-dimensional grid for each picture (frame) in a compressed video using a compression method to which a motion compensation method is applied;

Generating a histogram for the motion vector; And

Detecting camera motion from a distribution of bin values in the motion vector histogram; Its features are to include.

Here, in detecting the camera motion from the distribution of the bin values of the motion vector histogram, it is determined that the camera motion is generated when the total frequency of the motion vector histogram is greater than or equal to a predetermined threshold with respect to the total number of macroblocks. have.

Further, in detecting the camera motion, the total frequency of the motion vector histogram is greater than or equal to a predetermined threshold for the total number of macroblocks, the motion vector histogram is concentrated in a few adjacent bins, and the representative size of the bin is zero. If not, it is characterized in that it is determined that any one of the camera motion of the track, pan, boom, and tilt has occurred.

In addition, since the motion vector histogram is concentrated on a few adjacent bins, in the horizontal direction of the bin to which the motion vector (0, 0) is mapped, when the few bins are mapped, a camera motion of a track or a pan is generated. If the number of bins are mapped in the vertical direction of the bin to which the motion vectors (0, 0) are mapped, the camera motion of the boom or tilt is determined to be generated.

In addition, in detecting the camera motion, the total frequency of the motion vector histogram is greater than or equal to a predetermined threshold with respect to the total number of macroblocks, and the representative size of the surrounding bins is centered around the bin to which the motion vectors (0, 0) are mapped. When the values are mapped so as to be radially symmetrical, there is a feature in that it is determined that camera motion of dolly or zoom has occurred.

According to the present invention, a motion vector is divided into horizontal / vertical two-dimensional grids for each picture (frame) in a compressed video using a compression method to which a motion compensation method is applied, and a motion vector histogram is generated. As a result, the amount of data for the motion vector can be reduced, and a motion vector histogram containing direction and size information for the motion vector can be generated.

In addition, the present invention has the advantage that it is possible to quickly detect the camera motion from the distribution of the bin of the motion vector histogram through the analysis of the motion vector histogram generated using the motion vector quantization method of the video as described above.

In general, a motion vector of a compressed video using a motion compensation method such as MPEG-1 / 2, H.261, or H.263 is divided into 16x16 macroblocks. A horizontal / vertical two-dimensional vector that indicates where to refer to, reflecting object or camera motion.

Accordingly, the direction of the motion vector reflects the direction of the object / camera motion, and the size of the motion vector reflects the magnitude of the object / camera motion. Therefore, analyzing the motion vector value shown in one frame can quantify the amount or direction of motion. In the present invention, a method of quantizing the motion vector values and analyzing the motion vector histogram will be discussed.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

A motion vector histogram is an arbitrary level as a histogram of a motion vector for a P picture or a B picture to which a motion compensation method is applied, which is a two-dimensional histogram that is an arbitrary level in the horizontal and vertical directions. (E.g., as shown in FIG. 1, the histogram of the motion vector is 64 levels, which can be thought of as a two-dimensional histogram of 8 x 8 levels, 8 levels each in the horizontal / vertical direction).

At this time, in order to obtain the index of the motion vector histogram, as shown in FIG. 1, the motion vectors are quantized into a horizontal / vertical two-dimensional grid to obtain the number of motion vectors mapped to each grid. As described above, in order to quantize the motion vector, a maximum range of the motion vector must be determined in advance, which can be determined by the following method.

The first method is to determine the maximum range.

Specify the maximum range horizontal / vertical value of the motion vector to any sufficient value that can include the motion vectors that appear in a typical video. To do this, it is necessary to use a large number of test data images to find out an arbitrary range value sufficient to include motion vectors. It was appropriate when looking at the square region of (-128) ~ (+128) vertically.

The second method is to determine the maximum range using f_code.

The maximum range of motion vectors required for motion vector quantization may be determined by f_code, which is one of MPEG-1 / 2 encoded data in the case of a video compressed with MPEG-1 / 2.

Meaning of f_code in MPEG-1 F_code Maximum motion vector range 0 One 8 2 16 3 32 4 64 5 128 6 256 7 512 8 1024

Meaning of f_code in MPEG-2 F_code Maximum motion vector range One 8 2 16 3 32 4 64 5 128 6 256 7 512 8 1024 9 2048

In this case, f_code is a value for limiting the range of the motion vector for each picture, and the range of the motion vector according to the f_code value is as shown in [Table 1] and [Table 2]. Here, [Table 1] shows the meaning of f_code in MPEG-1, and [Table 2] shows the meaning of f_code in MPEG-2.

That is, if the f_code value of any picture of the MPEG-1 video is 4, the maximum range of the motion vector can be viewed as (-64) to (+64). In addition, if the f_code value of any picture of the MPEG-2 video is 4, the maximum range of the motion vector can be viewed as (-64) to (+64).

Next, the interval segmentation method in which the horizontal / vertical level of the motion vector is determined may be divided into a method of dividing the maximum range of the motion vector evenly, linearly, or nonlinearly by designating each division point. When the interval of the motion vector is divided in this way, the quantized shape of the motion vector can be expressed as shown in FIG. 2.

Meanwhile, the specific motion vector quantization method may be performed as shown in FIG. 3. In the example shown in FIG. 3, the maximum range of the motion vector is (-256) to (+256) in both the horizontal and vertical directions, and the quantization interval division is divided into eight equally divided horizontally and vertically, which is a total of 64 levels. The result is a quantization index from 0 to 63.

At this time, the index designation of each quantum can be as shown in Fig. 3, and in particular, in this quantization method, the motion vector (0, 0) is the center of the motion vector range for detecting the camera motion, and further, the quantum (lattice) In this case, the section is divided to be located at the center of the bin 36 (the description will be supplemented in the description of the camera motion detection).

For example, when there are motion vectors mv1 = (-128, 128), mv2 = (288, 160), and mv3 = (-96, -320), mv1 is mapped to quantum index 50 and exceeds the motion vector maximum range. mv2 and mv3 are mapped to quantum indices 55 and 3, approximating the nearest quantum.

On the other hand, the camera motion, as shown in Figure 4, there are a variety of motion, such as track (track), boom (dolly), dolly (pan), tilt (tilt) and roll (roll), MPEG- A motion vector of a compressed video using a compression method to which motion compensation methods such as 1/2, H.261, and H.263 are applied may provide information about camera motion. That is, when camera motion is generated, it is possible to detect that camera motion is generated by using the point that almost all motion vectors in a picture are affected (unlike the generation of object motion).

At this time, when camera motions of 'tracking', 'booming', 'panning' and 'tilting' occur among various camera motions, almost all motion vectors in the picture have a similar direction and size.

For example, when the motion vector in a picture is as shown in Fig. 5 (a), the motion vector histogram for this picture is expressed in two dimensions, as shown in Fig. 5 (b). Analyzing this histogram shows that when one bin has a very high bin value, when the bin is mapped to a two-dimensional quantized shape, the direction is horizontal right (say d) and is some predefined value (s) Since it is known that the motion vectors are empty, it can be seen that a camera 'tracking' or 'panning' having a direction 'd' and a size 's' has occurred in this picture.

When the motion vector in a picture is the same as that in FIG. 6 (a), the motion vector histogram for the picture is expressed in two dimensions as in FIG. 6 (b). Analyzing this histogram, one bin has a very high bin value, and when we map this bin to a two-dimensional quantization shape, the direction is vertically up (let d) and is some predefined value (s). Since the motion vectors are empty, it can be seen that a camera 'booming' or 'tilting' having a direction 'd' and a size 's' has been generated.

However, as shown in FIGS. 5 and 6, when the camera moves in a straight line such as 'tracking', 'panning', 'booming' or 'tilting', it is an ideal example that all motion vectors are mapped to only one bin. In general, most motion vectors are mapped to several adjacent bins. In this case, track / pan / boom / tilt information for camera motion can be detected by calculating a representative direction and size of adjacent bins.

On the other hand, when the camera motion is dolly or zoom, more complicated analysis is required to detect the camera motion.

That is, in the case of 'dolly forward' or 'zoom in', the motion vector of the picture is shown in FIG. 7 (a), and in case of 'dolly backward' or 'zoom out', the motion vector of the picture is shown in FIG. In both cases, the motion vector histogram is expressed in two dimensions as shown in FIG.

Analyzing this motion vector histogram, bins at the same distance from the bins mapped to the motion vectors (0, 0) (conventionally called 'center bins') will have similar bin values. That is, the bins indicated by the same pattern in FIG. 7C have similar bin values. This is because, as shown in FIGS. 7A and 7B, when 'dolly' or 'zoom' occurs in the camera motion, motion vectors of the same size are centered on the 'dolly center' or 'zoom center'. It appears to be radial.

However, in the quantization method of the present invention, since the motion vector (0, 0) is mapped to the center of the center bin, the result of quantizing the motion vectors appearing radially in the picture is also mapped to the bins radially around the center bin. . Therefore, compare the values of the upper, lower, left, and right bins (vertical pattern bins and horizontal pattern bins in the drawing) centering on the center bin, and check the bins in the diagonal direction (north-southwestern diagonal bins in the drawing, northwest -Compare the values of the southeast diagonal pattern bindles).

Thus, if you compare the values of the bins that are at the same distance from the center bins and the values are similar, and most of the macroblocks in the picture have these motion vectors (differences from object motion), then the current picture is 'dolly'. It can be determined that the camera motion of 'or' zoom 'has occurred.

Therefore, even if not implemented on a limited system such as a PVR, using the motion vector histogram and the camera motion detection method described in the present invention, it is possible to easily detect the camera motion compared to the method such as 'affine transform'. The generated motion vector histogram is a clue for generating an index of motion information of a video, and the following applications are possible.

First, the intelligent fast forward system of the video can automatically adjust the fast view speed by using the index information of how complex the motion of a specific section is, and the camera is zoomed in depending on the purpose. Zoom out or tracked shots may be indexed separately.

When the camera motion is detected, the error of the shot change alarm due to the camera motion can be eliminated in the shot change detection, and the excessive due to the camera motion in the intelligent fast forward. You can also remove the motion complexity, or slow down the view speed in a zoomed-in scene (assuming it's important, so it's zoomed in) so you can see more slowly.

As described above, according to the motion vector quantization method of a video according to the present invention, a motion vector is quantized into a horizontal / vertical two-dimensional lattice for each picture (frame) in a compressed video using a compression method to which a motion compensation method is applied. By dividing the interval and generating the motion vector histogram, it is possible to reduce the amount of data for the motion vector and generate a motion vector histogram containing the direction and the size information for the motion vector.

In addition, according to the camera motion detection method according to the present invention, through the analysis of the motion vector histogram generated by the motion vector quantization method of the video as described above, it is possible to quickly detect the camera motion from the distribution of the bin of the motion vector histogram There is an advantage.

Claims (13)

delete delete delete delete delete delete delete For each picture (frame) in the compressed video using the motion compensation method, the motion vector (0, 0) is centered in the quantization region, and the upper, lower, left, and right rectangles are applied to the maximum range of the specified motion vector. Dividing the quantization section into horizontal / vertical two-dimensional grids by setting a quantization section in the region; Generating a histogram for the motion vector by detecting the number of motion vectors mapped to the quantization intervals; And Determining from the distribution of the bin values of the motion vector histogram that the camera motion is generated if the total frequency of the motion vector histogram is equal to or greater than a predetermined threshold for the total number of macroblocks; Camera motion detection method using a motion vector quantization of a video comprising a. delete The method of claim 8, In determining that the camera motion has occurred , the total frequency of the motion vector histogram is greater than or equal to a predetermined threshold for the total number of macroblocks, the motion vector histogram is concentrated in a few adjacent bins, and the representative size of the bin is If not 0, the camera motion detection method using the motion vector quantization of the video, characterized in that it is determined that any one of the camera motion of the track, pan, boom, tilt . The method of claim 10, If the motion vector histogram is concentrated on a few adjacent bins and in the horizontal direction of the bin to which the motion vector (0, 0) is mapped, it is determined that the camera motion of the track or the pan has occurred. Camera motion detection method using a motion vector quantization of a video, characterized in that the . The method of claim 10, If the motion vector histogram is concentrated in a few adjacent bins and the bins are mapped in the vertical direction of the bin to which the motion vector (0, 0) is mapped, it is determined that a boom or tilt camera motion has occurred. Camera motion detection method using a motion vector quantization of a video, characterized in that the . The method of claim 8, In determining that the camera motion has occurred , the representative of neighboring bins is based on a bin whose total frequency of the motion vector histogram is equal to or greater than a predetermined threshold for the total number of macroblocks, and to which a motion vector (0, 0) is mapped. When the magnitude value is mapped to be radially symmetric , camera motion detection method using motion vector quantization of a video, characterized in that it is determined that camera motion of dolly or zoom has occurred .