JPH07112283B2 - Motion vector detection device and detection method - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION (Industrial field of application) The present invention is effective in various devices such as recording devices, transmission devices, and other display devices that perform signal processing of digital signals, so that moving image signals can be efficiently processed with a smaller code amount. The present invention relates to high-efficiency coding, and more particularly to a motion vector detection device and a motion vector detection method.

(Prior Art) Among the high-efficiency under-code schemes for encoding continuously input moving image signals with a smaller code amount, inter-frame predictive encoding is used as an encoding method that utilizes correlation between frames of image signals. There is.

This is because normal moving images are quite similar between each frame, so the signal of the frame to be encoded is predicted from the signal of the previous frame that has been added, and only the prediction error (residual error) is encoded. To do.

However, in the prediction from the pixels at the same position in the previous frame,
If the image is moving, the prediction error (residual error) increases, and the coding efficiency decreases. Therefore, the 2D spatial position of the previous frame used for prediction is moved (shifted) according to the movement of the image.
Then, motion compensation (correction) coding has been proposed in which inter-frame prediction is performed.

An example of this motion compensation coding is shown in FIG. 2, but a motion vector detector 12, a position shifter 11 and a motion vector variable length coder 13 are used for the configuration of a predictive coding device that does not perform motion compensation. Has been added.

<Predictive Coding Processing> In FIG. 2, a predictive signal (predictive value) is subtracted from a predictive signal subtractor 2 for a moving image signal continuously input from the image signal input terminal 1, and a predictive error (residual error) ) Is encoded. The method of forming the prediction signal will be described later.

Here, the prediction error (residual) may be quantized as it is, but in order to obtain higher coding efficiency, it is orthogonally transformed by the orthogonal cosine transform (DCT) by the orthogonal transformer 3 and then by the quantizer 4. It is generally quantized. Since the distribution of the quantized signal is concentrated in the vicinity of 0 (zero), it is converted into a variable length code such as a Huffman code by the variable length encoder 5 and output from the data output terminal 6 as variable length digital data. , Recorded or transmitted.

On the decoding device side, the variable-length digital data is converted into the original fixed-length data, replaced with the representative value by the inverse quantizer (representative value setting), and the inverse transform process of orthogonal transform is performed. Since this signal is a prediction error (residual error), it is added to the prediction signal to obtain a reproduced image signal.

On the other hand, the prediction signal in the transcoder needs to obtain the same signal as that in the decoder, and is generated from the quantized signal.
Therefore, in the encoding device of FIG. 2, the quantized signal is replaced with a representative value by the inverse quantizer 7 (representative value setting) as in the above-described decoding device, and further by the orthogonal inverse transformer 8. The inverse transform process of the orthogonal transform is performed.

Since the signal thus obtained corresponds to the decoded prediction error (residual error), the predicted signal of the preceding frame is added by the adder 9 to be a decoded image signal.
Further, this signal is delayed by one frame by the frame memory 10, and the position shifter 11 moves the position in the two-dimensional space by an amount corresponding to the movement, and becomes a prediction signal.

<Motion Compensation Prediction Process> The amount of movement of the image by the position shifter 11, that is, the motion vector is given by the motion vector detector 12.

Although various methods have been proposed for motion vector detection, the most common method is block matching. This is to divide an image into blocks of about 8 × 8 to 16 × 16, and obtain one motion vector in the unit. For example, a motion vector is two-dimensionally set within a range of about +3 (pixels / frame) to -3 (pixels / frame) for each pixel (hereinafter, referred to as pixel / frame) per frame, and is used for prediction. The image to be used is moved by that vector and the mean square error of the block is obtained. Then, the error is obtained for all the set vectors, and the vector with the smallest error is selected as the motion vector. Since the distribution of the selected motion vectors is concentrated near 0 (zero), it is converted into a variable length code such as Huffman code by the variable length encoder 13 and output from the motion vector data output terminal 14. To be done. On the other hand, on the decoding side, the position of the predicted image is shifted by this motion vector as in the coding system.

This method has a simple idea, but the amount of processing is extremely large because the error is calculated for each vector. Furthermore, the more the number of vectors, the more accurate the motion vector can be detected,
The throughput increases proportionally. In the previous example, processing is performed on 49 vectors with 7 values in both vertical and horizontal directions.

<Motion Vector Detection> As a means for reducing such a processing amount, a method of dividing vector selection into several steps has been proposed. this is,
First, coarsely set the vectors, then finely set around the selected vector.The number of vectors to be selected in each step is significantly different from the number of vectors set as a whole. Can be reduced. An example of vector setting in this case is shown in FIG.

In the figure, the first step is 1 (pixel / frame).
Every 9 vectors are selected (vectors indicated by ○ in the figure), and in the second step, 9 vectors are selected every 1 (pixel / frame) around the vector selected in the first step (indicated by ● in the figure). Vector). In this case, the total number of vectors is “49”, but the number of vectors at each step is “9”, and it is only necessary to obtain the error for the vector of “18”.

In this way, if the number of vectors in the second step is set to be the same as that in the first step, the processing in the first step and the second step only differs in the size of the set vector. Therefore, the same processing circuit can be used twice, and the processing circuit needs only one step.

(Problems to be Solved by the Invention) However, although such a conventional type of motion vector detection can reduce the processing amount with respect to the total number of vectors, the number of processing steps is increased and the processing amount is sufficiently reduced. Not. Moreover, since the number of vectors is not reduced, the amount of information that is variable-length coded and transmitted remains unchanged.

On the other hand, although the motion vector has a large inter-frame correlation, each frame is processed independently and the correlation is not used effectively.

Therefore, an object of the present invention is to provide a motion vector detection device and a motion vector detection method that solve the above-mentioned problems of the conventional technology.

(Means for Solving the Problem) The present invention is for performing inter-frame prediction by moving the spatial position of a frame used for prediction among continuous frames of image signals continuously input in accordance with the motion of an image. A first motion vector detecting means for setting a specific frame from the continuous frames of the image signal at a predetermined interval and detecting a motion vector of the image between the specific frames; A small amount of processing is performed by the second motion vector detecting means for detecting the motion vector between each frame of the specific frame and the specific frames before and after the specific frame in the vicinity of the motion vector detected by the first motion vector detecting means. A motion vector detecting device and a motion vector detecting method for detecting an accurate motion vector by quantity.

(Operation) In the motion vector detecting device having the above-described configuration, a specific frame is set in advance for every several frames from the continuous frames of the image signals continuously input, and the first motion is performed between the specific frames. The vector detecting means detects the motion vector in the first step.

Next, when detecting a motion vector between a non-specific frame between the specific frames and another specific frame, the second motion vector detecting means selects the second step as the second step in the first step. The motion vector is estimated based on the calculated motion vector, and motion vector detection with higher accuracy is performed around the motion vector of the estimation result.

(Example) An example of a motion vector detecting device and a motion vector detecting method according to the present invention will be described below with reference to the drawings.

FIG. 1 is a block diagram showing the configuration of an embodiment of an encoding device using a motion vector detecting device according to the present invention. The basic configuration of this encoding device conforms to the conventional example, and the same components as those in the above-mentioned FIG. 2 are denoted by the same reference numerals.

Referring to FIG. 1, the motion vector detecting device of the present invention comprises a motion vector detector 40, a vector difference detector 41, frame memories 32 and 33, (N-1) frame memory 31, a variable length encoder 13 and the like. It is composed of.

In FIG. 1, (N-) is used to encode a non-independent frame after the independent frame encoding used for prediction is completed.
1) It has a frame memory 31 [N is an integer of 2 or more].

Further, in order to form a predicted signal (predicted value) based on the two frames before and after, two frame memories 32 and 33, and a position shifter 34 that moves the spatial position (two-dimensional spatial position) of each signal. 35, a shift amount calculator 36 that gives the shifter amount to move with the position shifters 34, 35, and the position shifters 34, 35
Two coefficient multipliers {xα, x (1-α)} 37,3 that weight each signal whose spatial position has been moved by
There are 8 and their adders 39. [However, 0 <α <
1] Further, for detecting a motion vector, there are a motion vector detector 40 which is a first motion vector detecting means and a vector difference detector 41 which is a second motion vector detecting means.

Further, the changeover switch 42 is connected to the image signal input terminal 1 (N
-1) A changeover switch 43 is arranged between the frame memory 31 and the prediction signal subtractor 2 and the orthogonal switch 3, a changeover switch 44 is formed between the two frame memories 32 and 33, and a changeover switch 45 is formed between the frame memories 31 and 32. They are provided between the memory 33 and the vector difference detector 41, respectively.

In the configuration of the embodiment shown in FIG. 1, the moving image signal (continuous frame) input from the image signal input terminal 1 is
The changeover switches 42 and 43 are connected to the side a for independently encoded frames, and are guided to the orthogonal transformer 3 without passing through the (N-1) frame memory 31 or the prediction signal subtractor 2.

The operations of the orthogonal transformer 3, the quantizer 4, and the variable length encoder 5 are basically the same as those of the conventional example.

On the other hand, the remaining non-independent frames are inter-frame predicted, so the prediction signal is subtracted. However, in the device of the present invention, it is necessary to encode the independent frames first, so the remaining frames are delayed by that amount. .

Here, if the independent frame is one frame for N frames [N is an integer of 2 or more], the delay amount is (N−
1) It becomes the frame. Here, N is about 3 to 5. Changeover switch 4 for the remaining non-independent frames
2,43 is connected to the b side, the signal is (N-1) frame memory
In (31), the prediction signal is delayed by (N-1) frames, and the prediction signal is subtracted by the prediction signal subtractor 2 and then guided to the orthogonal transformer 3, where the prediction error (residual error) is encoded.

Here, the changeover switches 42 and 43 are periodically connected to the a side for one N frame, and are connected to the b side otherwise. Subsequent operations of the orthogonal transformer 3, the quantizer 4, and the variable length encoder 5 are the same as those in the independent frame.

The state of the above-described predictive coding process is shown in FIG. 7, where a is a conventional interframe prediction method and b is an interframe prediction method in the case of the present invention. In the figure, a square is a continuous frame of the same image signal that is continuously input, and a shaded part is a frame that is independently coded in the frame, and the first in a (or reset). Time)
Only b is an independent frame, but b has periodic independent frames. Arrows indicate the directional relationship of inter-frame prediction. In a, prediction is performed only from the previous frame as in each frame, but in b, prediction is performed from the two preceding and following independent frames.

In addition, the prediction is performed only on the basis of the independent frames, and the frames predicted from the previous and subsequent frames are not used for another prediction as in the case of b.

<Prediction signal formation> Next, a method of forming a prediction signal in the method of the present invention will be described.

The prediction signal is obtained from the signal of the new independent frame stored in the frame memory 32 and the signal of the old independent frame stored in the frame memory 33.

A signal is input to the frame memory 32 during the independent frame processing, and at the same time, the changeover switch 44 is connected to the side a so that the signal of the frame memory 32 is replaced with the frame memory 33. During the prediction process, these outputs are moved (shifted) by the shift amounts given by the shift amount calculator 36 by the position shifters 34 and 35 (two-dimensional spatial positions).

In the shift amount calculator 36, the input motion vector is (-i) for the old frame from the time relationship between the frames.
The new frame is multiplied by (N-i) times to obtain the shift amount.

Here, i is the number of the prediction frame in which the independent frame is 0 (zero), and i = 1, 2, 3, ..., (N−i).

The outputs of the position shifters 34 and 35 are multiplied by α by the coefficient multiplier 37 and 38 by the coefficient multiplier 37 and 38, and (1 by the coefficient multiplier 38.
-[Alpha]) times [where 0 <[alpha] <1], added by the adder 39, and supplied to the prediction signal subtractor 2 as a prediction signal. Here, the weighting coefficient α is
i / N.

<Motion Vector Detection> Next, in the motion vector detecting method of the present invention, the process of the first step is performed by the first motion vector detecting means, that is, the motion vector detector 40 shown in FIG. An example of the structure of the motion vector detector 40 is shown in FIG.

In FIG. 3, the signals of the two frames for which the mean square error is obtained are the signal of the new frame from the vector detection signal input terminal [A] 51 and the signal of the old frame from the vector detection signal input terminal [B] 52. It is input and written in the random access memory (RAM) 53, 54.

Here, the spatial area of the signal held in the RAMs 53 and 54 is set larger than the spatial area moved by the vector in which the block of 8 × 8 or 16 × 16 pixels for which the motion vector is obtained is set.

Since the detected motion vector is used in a non-independent frame sandwiched between two input frames, both frames are moved symmetrically to obtain the motion vector.

The addresses given to the RAMs 53 and 54 are obtained by adding / subtracting the address shift amount to / from the main address at the time of reading and the main address at the time of reading.

Moreover, although the write operation is performed only once, the read operation is performed by the number of vectors. As the address shift amount, the set vector value corresponding to N frames output from the vector value generator 55 can be used as it is. That is, the address of the RAM 53 corresponding to the new frame is obtained by adding the vector value output from the vector value generator 55 to the output of the main address generator 56 by the adder 57, and the address of the RAM 54 corresponding to the old frame is A subtracter 58 subtracts the vector value from the output of the address generator 56.

The outputs of the RAMs 53 and 54 thus obtained are the subtractor 59.
The difference is obtained in step S1, and the square value is converted into the square value of the input value in the squarer 60, and cumulative addition is performed for one block in the integrator 61 to obtain the mean square error for the set vector value.

Here, since the processing of the mean square error detection (calculation) unit surrounded by a broken line in FIG. 3 is repeated for each set vector in each block by the number of pixels in the block, the processing amount is the number of input pixels. It is a vector multiple of the number. In other words, when processing is performed in real time, the processing cycle is multiplied by the number of vectors of the pixel cycle, and when the set number of vectors is "9", processing speed of 9 times is required. However, in low-rate coding such as a videophone, the number of pixels is reduced from 1/4 to 1/16 of that of a normal TV signal, and processing at about 9 times that speed is not a problem. On the other hand, when the pixel rate is high, parallel processing is performed for each vector, and the circuit amount increases in proportion to the number of vectors.

The mean squared error of each vector thus obtained is
The smallest one is selected by the vector selector 62, and the motion vector value V ′ for one frame of the vector is output. The selected motion vector value V'is stored in the frame memory 63 because it is used during the processing of the prediction frame.
It is output from the vector value output terminal 64.

Here, since the motion vector value is generated for each block,
The frame memory 63 is a frame memory for pixel values such as 32 or
A small capacity of 1 / (number of pixels in a block) with respect to 33 is sufficient.

<Vector Difference Detection> Next, the second step is a process for obtaining a vector difference in each prediction frame, which is performed by the second motion vector detecting means, that is, the vector difference detector 41 shown in FIG. Be done. An example of the construction of the vector difference detector 41 is shown in FIG. 4. The difference from the motion vector detector 40 shown in FIG. 3 is that the mean square error surrounded by a broken line in FIG. The detection unit is the same as in FIG.

For the two frame signals for which the mean square error is obtained, the predicted frame signal is from the vector detection signal input terminal [A] 51, and the old and new (previous and subsequent) independent frame signals are from the vector detection signal input terminal [A]. B] 52, respectively, and written in the RAMs 53, 54.

Whether the old frame or the new frame (previous or posterior) is used as an independent frame is selected as a signal of a frame that is weighted more in prediction. That is, the output of the frame memory 32, which is a signal of the new frame, and the output of the frame memory 33, which is a signal of the old frame, are periodically changed by the connection switch on the c side and the d side by the changeover switch 45 according to the frame number. To be selected.

To generate the vector value, first, the motion vector value V'between independent frames output from the vector value output terminal 64 shown in FIG. 3 is the vector value input terminal 65 shown in FIG.
Input from the vector difference generator 66 and adder
The vector values are obtained by adding at 67. Here, dV is considered similar to the vector difference in the second step of the conventional example.

Since this vector value is the amount per frame, the actual address shift amount depends on the time relation of the frame.
Then, the old frame is multiplied by (-i) and the new frame is multiplied by (N-i). The obtained address shift amount is an estimated value of the motion vector of the predicted frame,
RA is added as the read address for the old or new (previous or subsequent) independent frame by the adder 69 with the main address.
Input to M54. On the other hand, since the predicted frame is considered fixed, only the main address is used.

The mean square error of each vector is calculated from the output of each RAM 53, 54 by the subtracter 59, the squarer 60, and the integrator 61, and it becomes the minimum.
The value dVS of dV is obtained by the vector selector 62, and the final vector value V is obtained by adding it to the motion vector value V'in the adder 70.

The motion vector value V ′ and the value dV are 9 kinds, but the vector value V is selected from 49 vectors.

With the above processing, the processing of the motion vector detector 40 shown in FIG. 3 and the processing of the vector difference detector 41 shown in FIG. 4 are not performed at the same time. Also, since the mean square error detectors that require high-speed operation are almost the same, that part can be integrated, and the circuit amount is not much different from one motion vector detector.

As described above, in FIG. 1 described above, the input image signal is obtained as independent frames every several frames by the changeover switches 42 and 43, and the motion vector detector 40 detects the motion vector between the independent frames. Desired.

On the other hand, the signal of the non-independent frame is delayed by the (N-1) frame memory 31, and the image signal of the independent frame used for prediction is processed by the frame memories 32 and 33 before and after the prediction processing of the non-independent frame. Two frames (before and after) are stored. The output of the frame memory 32 or 33 is selected by the changeover switch 45 and given to the vector difference detector 41 which is the second motion vector detecting means.

The vector difference detector 41 performs the second step motion vector detection with the predicted frame signal.

As described above, in the method of the present invention, the frame subject to the motion vector detection processing changes for each processing step,
In each processing of each frame, only one step is processed. This situation is shown in FIG. 6. In the conventional example of a, the processing of both the first and second steps is performed in each frame, whereas in the case of the method of the present invention of b, an independent frame (shadow is added). Frame), the first step is performed, and in the predicted non-independent frame, only the second step is performed.

Although the number of steps is two in this embodiment,
A higher number of stages is also conceivable. In that case, the processing is such that the first step and the second step are performed in the independent frame, and the third step and the fourth step are performed in the prediction frame.

Further, in the above-described embodiment, independent frames that are independently coded within a frame without using inter-frame prediction from among consecutive frames of image signals that are continuously input are set every few frames in advance, and The first step is a rough motion vector detection between frames, and then the second step is a highly accurate vector difference detection.
The present invention is not limited to this, and can also be applied to motion vector detection in conventional coding by inter-frame prediction in which independent frames are not set as described above.

Further, since the present invention is applied to one image unit, it is needless to say that the present invention is applied to a frame in a television signal, but it is needless to say that the present invention can be applied to not only this but also a field. .

(Effects of the Invention) As described above, in the present invention, a plurality of specific frames are set in advance from the continuous frames of image signals that are continuously input, and the first motion vector detecting means is provided between the specific frames. When the rough motion vector detection in the first step is performed and then the motion vector is detected for the non-specific frame between the specific frames, the second motion vector detecting means performs the second step as the second step. A motion vector between the non-specific image other than the specific image and the specific image is estimated based on the motion vector selected in one step, and the motion between the non-specific image and the specific image is around the estimated motion vector. Since the vector detection is performed with high accuracy, the number of processing steps in each frame can be reduced. In the method, two frames are processed in each frame, whereas the amount of processing is only one half, that is, one step.

The vector of the first step by the first motion vector detecting means is common at a predetermined interval (several frames), and the amount of information of the vector that needs to be transmitted to the decoding side is reduced. This is because the processing steps are performed over several frames, so that the inter-frame correlation of motion vectors, which has not been used in the conventional method, is effectively used.

On the other hand, although the vector range is limited in the second step by the second motion vector detecting means, it does not cause a problem because the inter-frame correlation of the motion vector is high, and rather the range is limited, so that it is erroneous due to noise or the like. The selected vector is less likely to be selected, and good vector detection is performed.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of an embodiment of an encoding device using a motion vector detecting device according to the present invention, FIG. 2 is a block diagram showing a configuration of a motion compensation encoding device in a conventional example, and FIG. FIG. 4 is a block diagram showing a first motion vector detecting means constituting an embodiment of the present invention, FIG. 4 is a block diagram showing a second motion vector detecting means constituting an embodiment of the present invention, and FIG. FIG. 6 is a diagram showing an example of setting a motion vector in a two-step process, FIG. 6 is a diagram showing processing steps in each frame of the present invention and a conventional example, and FIG. 7 is a diagram showing a prediction method of the present invention and a conventional example. . 1 ... Image signal input terminal, 2 ... Prediction signal subtractor, 3 ... Orthogonal transformer, 4 ... Quantizer, 5,13 ... Variable length encoder, 6 ... Data output terminal, 7 ... … Inverse quantizer, 8 …… Orthogonal inverse transformer, 9,39,57,67,69,70 …… Adder, 10,32,33,63 …… Frame memory, 11,34,35 …… Position Shifter, 12 …… Motion vector detector, 14 …… Motion vector data output terminal, 31 …… (N-1) frame memory, 36 …… Shift amount calculator, 37,38 …… Coefficient multiplier, 40 …… First motion vector detecting means (motion vector detector), 41 ... Second motion vector detecting means (vector difference detector), 42,43,44,45 ... Changeover switch, 51,52 ... Vector detection Signal input terminal, 53,54 …… Random access memory (RAM), 55 …… Vector value generator, 56 …… Main address generator, 58, 59 …… Subtractor, 60 …… Squarer, 61… … Integrator, 62 …… Vector selector, 64 …… Vector value output terminal, 65 …… Vector value input terminal, 66 …… Vector difference generator, 68 …… Multiplier.

Claims (5)

[Claims] 1. A motion vector detection device for detecting a motion vector when performing motion compensation inter-picture prediction for image signals input continuously, wherein a plurality of specific images are set from among continuous input images, First motion vector detecting means for detecting a motion vector between the specific images, and a motion vector between the non-specific image other than the specific image and the specific image is estimated based on an output of the first motion vector detecting means. And detecting a motion vector between the non-specific image and the specific image around the estimated motion vector.
Motion vector detecting device comprising the motion vector detecting means of FIG. 2. The motion vector detecting device according to claim 1, wherein the second motion vector detecting means sets the motion vector between the non-specific image and the specific image to the first motion vector. A motion vector detecting device characterized in that the motion vector is estimated based on the output of the detecting means and the motion vector is detected as a difference from the estimated motion vector. 3. A motion vector detection method for detecting a motion vector when performing motion-compensated inter-picture prediction for image signals that are continuously input, wherein a plurality of specific images are set from among continuously input images. The motion vector between the specific images is detected, the motion vector between the non-specific image other than the specific image and the specific image is estimated based on the motion vector between the specific images, and the motion vector around the estimated motion vector A motion vector detecting method characterized in that a motion vector between a non-specific image and the specific image is detected. 4. The motion vector detecting method according to claim 3, wherein the motion vector between the non-specific image and the specific image is estimated based on the motion vector between the specific images. A motion vector detecting method characterized in that estimation is performed according to a temporal relationship between a specific image and the non-specific image. 5. The motion vector detecting method according to claim 3, wherein the motion vector between the non-specific image and the specific image is estimated based on the motion vector between the specific images, and the estimation is performed. A motion vector detecting method characterized in that a motion vector is detected as a difference from the motion vector.