JP2000059786A - Device and method for detecting motion - Google Patents

Abstract

PROBLEM TO BE SOLVED: To detect a motion vector of an image for which the motion about the image as a whole is large by detecting a motion vector about plural macroblocks, calculating a motion vector regarding the image as a whole by using the motion vector and calculating a motion vector about each macroblock by using the motion vector regarding the image as a whole. SOLUTION: A global vector detecting part 5 extracts plural macroblocks from the image data of inputted on frame and detects a motion vector about each extracted macroblock. Furthermore, the part 5 calculates an evaluation function using these motion vectors and generates a global vector. Meanwhile, image data is inputted to a motion detecting part 3 from a frame buffer 2 for each macroblock and the part 3 performs pattern matching about each macroblock by using the global vector, detects a motion vector MV regarding each macroblock, and outputs it.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] 1. Field of the Invention [0002] The present invention relates to a motion detecting apparatus and method for performing a motion detecting process performed when encoding image data representing a moving image by MPEG (Moving Picture Experts Group).

[0002]

2. Description of the Related Art As a conventional moving picture coding apparatus for coding image data representing a moving picture, a hybrid coding which combines motion compensated inter-frame prediction and DCT (Discrete Cosine Transform) has been proposed. × 8
In some cases, this is performed in units of macroblocks composed of pixels.

[0003] This moving picture coding apparatus detects a motion vector between frames, and performs motion compensation using the detected motion vector, thereby reducing the amount of code at the time of coding.

There are various methods for performing motion detection in such a conventional moving picture coding apparatus. The starting point for performing motion detection is that macroblocks existing at corresponding positions between frames are used. Often compared. When comparing macroblocks, it is generally unclear in which direction the image is moving. Therefore, macroblocks within a predetermined range are searched around the corresponding position between frames. By doing so, a macroblock having a small difference in luminance value is specified.

In addition, when a camera device for capturing a moving image is fixed at a predetermined position and the subject is moving, the direction in which the subject moves varies depending on the position of the entire image. Therefore, a search is often performed for macroblocks located in a wide range in all directions around the starting point. Here, when the movement of the subject is large, the search range may be out of the range. In such a case, it is dealt with by performing the encoding process from the inter macro block to the intra macro block. Switching from an inter macroblock to an intra macroblock and performing the encoding process is based on the premise that the motion of the subject is large and often does not fall outside the search range.

Further, in a conventional moving picture coding apparatus, a technique called a hierarchical search has been proposed in order to reduce the amount of calculation when performing motion detection. This technique is
In this method, motion detection is performed so as to detect one motion vector from a plurality of macroblocks, and when obtaining a motion vector for each macroblock, a motion vector for a plurality of previously detected macroblocks is used.

[0007]

However, the moving picture coding apparatus described above detects a macroblock having a small difference by searching for a macroblock in a predetermined range, thereby detecting a motion vector. In order to cope with a large motion, it is necessary to widen the search range, and the processing time required for motion detection exponentially increases.

[0008] In the above moving picture coding apparatus,
If the motion of the subject is large and the coding process is performed by switching from the inter macro block to the intra macro block, if the motion of the captured image is a motion outside the search range, all macro blocks are changed to the intra macro block. Processing as a block increases the amount of computation and further increases the load on the device itself. This occurs, for example, when the entire image moves in a predetermined direction when the subject is imaged using a technique such as panning, and exceeds the search range.

Further, in a moving picture coding apparatus which obtains a motion vector for a plurality of macroblocks and obtains a motion vector for each macroblock, it is possible to detect a motion vector when there is a large motion outside the screen. It is not possible to obtain motion vectors for a plurality of macroblocks.

Accordingly, the present invention has been proposed in view of the above-described circumstances, and provides a motion detecting apparatus and method capable of detecting a motion vector of an image having a large motion on the entire screen. With the goal.

[0011]

A motion detecting apparatus according to the present invention for solving the above-mentioned problems comprises an extracting means for extracting a plurality of macroblocks from an image, and a motion for the plurality of macroblocks extracted by the extracting means. First motion detecting means for detecting a vector, motion calculating means for calculating a motion vector for the entire image using a motion vector for each macroblock detected by the first motion detecting means, and motion calculating means And a second motion estimator for calculating a motion vector for each macroblock using the motion vector calculated in (1).

[0012] The motion detection apparatus thus configured detects a motion vector for each macroblock using a motion vector for a plurality of macroblocks extracted from an image.

Further, in the motion detection method according to the present invention, a plurality of macroblocks are extracted from an image, motion vectors of the extracted plurality of macroblocks are detected, and a motion vector of each of the macroblocks is used. A motion vector for the entire image is calculated, and a motion vector of each macroblock is calculated using the calculated motion vector for the entire image.

In such a motion detection method, a motion vector for each macroblock is detected using motion vectors for a plurality of macroblocks extracted from an image.

[0015]

Embodiments of the present invention will be described below in detail with reference to the drawings.

An image coding apparatus 1 to which the present invention is applied is configured, for example, as shown in FIG. That is, the image encoding apparatus 1 includes a frame buffer 2 to which image data is input, a motion detection unit 3 that detects a motion component of the image data stored in the frame buffer 2, and a motion residual information A
D, a residual information generating unit 4 for generating D, a global vector detecting unit 5 for detecting a motion vector in the entire image,
A control unit 6 that outputs parameters and the like for performing encoding to each unit.

The frame buffer 2 receives image data from the outside and stores the image data for each frame. The frame buffer 2 is controlled to output image data at a predetermined timing, and outputs the image data to the motion detection unit 3, the residual information generation unit 4, the global vector detection unit 5, and the arithmetic unit 7.

The global vector detector 5 samples a plurality of macroblocks by performing sampling processing on the image data from the frame buffer 2, and detects a motion vector for the entire sampled macroblock. That is, the global vector detection unit 5 obtains a motion vector (global vector) for the entire screen by obtaining a motion vector for a plurality of entire macroblocks, and outputs the global vector to the motion detection unit 3.

Specifically, the global vector detector 5
Extracts, for example, a plurality of macroblocks located at different positions in one screen as shown in FIG. 2, and detects one motion vector in the entire extracted macroblocks. At this time, the global vector detection unit 5 obtains the global vector by applying the extracted motion vector for each macroblock to the evaluation function.
The global vector detection unit 5 obtains a global vector by using, as an evaluation function, a function formula of, for example, taking an average of motion vectors for each extracted macroblock.

Further, the global vector detecting section 5 may extract a plurality of adjacent macro blocks to obtain a global vector. That is, the global vector detecting unit 5 extracts not only a macroblock consisting of 8 × 8 pixels to obtain a global vector but also extracts an image of a small area consisting of 16 × 16 pixels and You may ask.

Further, the global vector detecting section 5 may obtain a global vector for each of the regions A, B and C obtained by dividing one screen in the vertical direction as shown in FIG. . Accordingly, the global vector detection unit 5 may perform, for example, a case in which the region A is an image showing a mountain photographed at a long distance, and the region C is an image showing a flower photographed in a short distance. A global vector can be obtained for each region even if the image has different motions on the screen. Note that the areas obtained by dividing one screen may overlap each other.

The motion detecting section 3 detects a motion vector MV in units of macroblocks of 8 × 8 pixels in the image data stored in the frame buffer 2. The motion detection unit 3 detects a motion vector MV by performing pattern matching between a macroblock constituting a reference frame and a macroblock read from the frame buffer 2, for example, and generates the detected motion vector MV as residual information. Output to the unit 4 and the control unit 6. At this time, the motion detection unit 3 generates a motion vector MV using the global vector from the global vector detection unit 5. That is, when performing the pattern matching of the motion vector MV for each macroblock in a predetermined search range, the motion detection unit 3 obtains the motion vector MV by changing the search range using the global vector as an offset. The motion detection unit 3 performs pattern matching by changing the center position of the search range according to the global vector.

The residual information generator 4 receives the motion vector MV from the motion detector 3 and the image data from the frame buffer 2 in macroblock units. The residual information generation unit 4 generates the sum of absolute values of the differences between the motion components as the motion residual information AD using the input motion vector MV and the image data, and outputs the motion residual information AD to the control unit 6. Output to

The control unit 6 determines a macroblock type when performing an encoding process using the motion vector MV from the motion detection unit 3 and the motion residual information AD from the residual information generation unit 4. The control unit 6 determines whether each macroblock is an inter macroblock according to a picture type,
It is determined whether the block is an intra macro block. Here, the inter macroblock is a macrobook encoded using the above-described motion residual information AD, and the intra macroblock is formed by using an average value separation residual indicating variation in luminance without using a motion component. This is the macroblock to be encoded.

The control section 6 generates control information for controlling the opening and closing operations of the switches 17 and 18 according to the determined macro block type. Further, the control unit 6
Outputs the motion vector MV from the motion detecting unit 3 to the motion compensating unit 16.

The image encoding apparatus 1 further includes an arithmetic unit 7 to which an image signal is input from the frame buffer 2, a DCT processing unit 8 for performing DCT (Discrete Cosine Transform) processing on the image data, A quantization processing unit 9 for performing a quantization process on the DCT coefficient from the DCT processing unit 8; a variable length encoding unit 10 for performing a variable length compression process on the DCT coefficient from the quantization processing unit 9; ,
Buffer 1 for storing the encoded image data
1 is provided.

The DCT processing unit 8 performs a two-dimensional DCT of 8 × 8 pixels on the image data of each macroblock input from the arithmetic unit 7 and outputs the obtained DCT coefficients. This D
The CT processing unit 8 outputs the DCT coefficients to the quantization processing unit 9.

The quantization processing unit 9 performs a quantization process on the DCT coefficient from the DCT processing unit 8 at a quantization scale changed for each macroblock. This quantization processing unit 9
Is the variable-length encoding unit 1
0 and output to the inverse quantization processing unit 12. Further, the quantization processing unit 9 may perform a quantization process and generate a CBP (Coded Block Pattern) of the input DCT coefficient. When generating the CBP, the quantization processing unit 9 outputs information indicating the CBP to the variable length coding unit 10.

The variable-length coding unit 10 receives the DCT coefficient from the quantization processing unit 9, the skip control signal from the skip control unit 10, and the motion vector MV from the control unit 6, and uses these information to perform coding. Perform the conversion process. The variable-length encoding unit 10 performs variable-length encoding based on the syntax of MPEG, performs header processing, code generation, and the like, generates image data, and outputs the encoded image data to the buffer 11.

The buffer 11 stores the image data from the variable length coding unit 10, and outputs the bit data with the timing controlled.

The image encoding apparatus 1 further includes an inverse quantization processing unit 12 for performing an inverse quantization process on the DCT coefficient from the quantization processing unit 9 and an inverse DT for the DCT coefficient from the inverse quantization processing unit.
An inverse DCT processing unit 13 for performing a CT process, a computing unit 14 to which the image data from the inverse DCT processing unit 13 is input, a buffer 15 for storing the image data, and a motion compensation process for the image data stored in the buffer 15 Motion compensator 16 for applying
And

The inverse quantization processing unit 12 performs an inverse quantization process on the DCT coefficient from the quantization processing unit 9. The inverse quantization processing unit 12 performs an inverse quantization process using the quantization scale obtained when the quantization processing unit 9 performs the quantization process, and outputs DCT coefficients to the inverse DCT processing unit 13.

The inverse DCT processing unit 13 includes the inverse quantization processing unit 1
2 is subjected to inverse DCT processing to the DCT coefficient from
Output to The arithmetic unit 14 performs the inverse DT processing in the inverse DCT processing unit 13.
Image data subjected to CT processing is input, and
The motion-compensated image data is input via the switch 17, and the motion-compensated image data and the inverse DC
The image data from the T processing unit 13 is added and output to the buffer 15.

The buffer 15 receives image data from the computing unit 14 and stores the input image data. The buffer 15 reads out predicted image data when the stored image data is subjected to motion compensation by the motion compensation unit 16.

The motion compensator 16 reads predicted image data for each macroblock from the buffer 15 based on the motion vector MV. The motion compensator 16 calculates the sum of absolute values of the difference between the motion vector MV from the controller 6 and the predicted image data for each macroblock.

In such an image encoding apparatus 1, I (Intra) -P
When generating an image, the image data stored in the frame buffer 2 is subjected to DCT processing and quantization processing by a DCT processing unit 8 and a quantization processing unit 9 via an arithmetic unit 7 for each macroblock, and is subjected to variable length coding. The data is encoded by the unit 10 and generated as a bit stream via the buffer 11.
Here, the signals that have been subjected to the above-described processing in the quantization processing unit 9 and the variable-length coding unit 10 are output to the inverse quantization processing unit 12 and the inverse D
The image data is formed again by the CT processing unit 13 and temporarily stored in the buffer 15.

In the image encoding device 1, P (Predictive)-
When a Picture is generated, a motion component MV is generated by a motion detection unit 3 by detecting a motion component of image data stored in a frame buffer 2, and a motion residual information AD is generated by a residual information generation unit 4. I do. Then, the motion vector MV is input to the motion compensation unit 16 via the control unit 6, and the motion compensation unit 16 performs a motion compensation process on the image data stored in the buffer 15 when generating the above-described I-Picture. Thus, predicted image data is generated. At this time, when motion compensation is performed by the motion compensator 16, the motion compensation is performed in macroblock units, and the switch 18 is closed and the switch 17 is opened by a switch control signal from the controller 6. Then, the motion compensation unit 1
The prediction image data output from 6 is subjected to a subtraction process with the image data stored in the frame buffer 2 by the arithmetic unit 7 via the switch 18, and the DCT processing unit 8 and the quantization processing unit 9 perform the same processing as described above. The data is processed, encoded by the variable-length encoding unit 10, and output from the buffer 11 as a bit stream.

Further, in the image encoding device 1, B (Bidirectiona
When generating (lly predictive) -Picture, the motion compensation unit 16 uses the image data of the frame stored in the buffer 15 before the encoding process and the image data of the previous frame to perform bidirectional motion compensation between frames. To generate predicted image data. Then, the predicted image data is subjected to a subtraction process from the image data stored in the frame buffer 2 by the arithmetic unit 7, and the same processing as described above is performed by the DCT processing unit 8 and the quantization processing unit 9 to obtain a variable length code. The data is encoded by the encoding unit 10 and output from the buffer 11 as a bit stream.

Next, an example of detecting the motion vector MV by the above-described image encoding apparatus 1 will be described with reference to the flowchart shown in FIG.

According to this flowchart, first, in step S1, one frame of image data is input to the frame buffer 2. Note that, in the description of the processing shown in FIG.
Up to this is the processing for detecting the global vector, and step S5 shows the processing for generating the motion vector MV for each macroblock in the motion detection unit 3.

In step S2, the global vector detection unit 5 receives the image data stored in the frame buffer 2 in units of one frame, and extracts a plurality of macroblocks from the input image data of one frame, for example, as shown in FIG. Extract as described above. Further, in this step S2, as shown in FIG. 3 described above, one screen may be divided into a plurality of regions and a plurality of macroblocks may be extracted for each region.

In step S3, the global vector detector 5 detects a motion vector for each macro block extracted in step S2.

In step S4, the global vector detecting unit 5 calculates an evaluation function using the motion vector for each macroblock obtained in step S3, and generates a global vector. Here, the global vector detection unit 5 generates a global vector by, for example, averaging the motion vectors for each macroblock as described above.

In step S5, the motion detecting section 3
Image data is input from the frame buffer 2 for each macroblock, and motion detection is performed by performing pattern matching for each macroblock using the global vector obtained in step S4 described above. At this time, the motion detection unit 3 performs pattern matching by changing the center position of the search range according to the global vector.

In step S6, the motion detecting section 3
A motion vector MV for each macroblock is detected based on the result of the motion detection performed in step S5, and the motion vector is output to the residual information generation unit 4 and the control unit 6.

In such an image coding apparatus 1, before the motion detecting section 3 generates a motion vector MV for each macroblock, the global vector detecting section 5 indicates a global motion vector indicating one motion vector for one entire screen. Since the vector is detected, the search range when detecting the motion vector MV for each macroblock in the motion detecting unit 3 does not need to be wide, and the process of detecting the motion vector MV can be reduced. That is, according to the image encoding device 1, for example, even for an image captured by a technique such as panning in which the entire screen is moving, the global vector detecting unit 5 obtains a global vector for the entire screen, It is possible to avoid a process of detecting a motion vector for each macroblock with a wide range.

Further, according to the image encoding apparatus 1, a search can be easily performed even for a high-speed movement on the entire screen which cannot be searched by the conventional motion detection method.

Further, according to the image encoding apparatus 1, by dividing the inside of one screen into a plurality of parts and calculating the global vector by the global vector detecting unit 5, an area having a large motion within one screen can be efficiently used. The motion vector MV can be detected.

[0049]

As described above in detail, according to the motion detecting apparatus and method according to the present invention, a plurality of macroblocks are extracted from an image, and motion vectors of the extracted plurality of macroblocks are detected. The motion vector for the entire image is calculated using the motion vector for each macroblock, and the motion vector for each macroblock is calculated using the calculated motion vector for the entire image. Before calculating a motion vector, a motion vector for the entire screen can be obtained. Therefore, according to the motion detection device and method according to the present invention, for example, a motion vector of an image having a large motion can be easily detected over the entire screen, and the processing for motion detection can be greatly reduced. .

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating an example of an image encoding device to which the present invention has been applied.

FIG. 2 is a diagram for explaining a process of extracting a macroblock when a global vector is detected by a global vector detection unit included in the image encoding device.

FIG. 3 is a diagram for describing a process of extracting a macroblock by dividing one screen into a plurality of regions when obtaining a global vector by a global vector detection unit included in the image encoding device.

FIG. 4 is a flowchart for describing processing when a motion detection process is performed by the image encoding device.

[Explanation of symbols]

1. Image encoding device, 3. Motion detector, 5. Global vector detector

Claims (6)

[Claims] 1. An extracting unit for extracting a plurality of macroblocks from an image, a first motion detecting unit for detecting a motion vector for the plurality of macroblocks extracted by the extracting unit, and a first motion detecting unit A motion calculating means for calculating a motion vector for the entire image using a motion vector for each macroblock detected in the step, and a motion vector for each macroblock using the motion vector calculated by the motion calculating means. A motion detection device comprising: two motion detection means. 2. The motion detecting apparatus according to claim 1, wherein said extracting means extracts a plurality of adjacent macro blocks. 3. The motion detecting apparatus according to claim 1, wherein said extracting means extracts a plurality of macroblocks from each area obtained by dividing the entire image. 4. A method for extracting a plurality of macroblocks from an image, detecting motion vectors for the plurality of extracted macroblocks, and calculating a motion vector for the entire image using the motion vectors for each of the macroblocks. A motion detection method, comprising: calculating a motion vector of each macroblock using a calculated motion vector for the entire image. 5. The motion detection method according to claim 4, wherein a plurality of adjacent macro blocks are extracted. 6. The method according to claim 4, wherein a plurality of macroblocks are extracted from each area obtained by dividing the entire image.
The motion detection method described.