JPH07240923A - Image encoding method - Google Patents

Abstract

PURPOSE:To prevent a motion vector from being erroneously detected and to reduce the amount of generated codes. CONSTITUTION:The absolute value sum of difference between a block A of a present frame and a block A' of a preceding frame is calculated at a differentiator 3 and a cumulative adder 4, for example, and this is defined as residual. On the other hand, a weight calculator 13 or 14 respectively calculates weight W1 or W2 based on the motion vector detected in the past. Then, a computing element 5 calculates the weighted residual, for which the weight W1 and W2 is multiplied to the residual, and a vector indicating the position of the block A' to apply the minimum weighted residual from the position of the block A is defined as the motion vector at a minimum value detection circuit 6.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention is suitable for use in, for example, a video conference system which encodes a moving image and transmits or records it in a recording medium, a video telephone system, a broadcasting device, a disk device, a tape device or the like. The present invention relates to an image coding method.

[0002]

2. Description of the Related Art In a system for transmitting a moving image signal to a remote place such as a conventional video conference system or a video telephone system, for example, an image signal is transmitted to a line in order to efficiently use a transmission line. The compression coding is performed by using the correlation and the inter-frame correlation.

When the line correlation is used, the image signal can be compressed by an orthogonal transform process such as a DCT (discrete cosine transform) process.

When the inter-frame correlation is used, the image signal can be further compressed. That is, for example, as shown in FIG. 8, times t1 and t2 (however,
For example, when t1 <t2, and t2-t1 is a value equal to the frame period), the frame image PC1,
When each PC2 is generated, for example, the difference of PC2 with respect to the frame image PC1 is calculated, and the difference image PC
12 is generated. Then, the difference image PC12 is encoded instead of the frame image PC2.

Usually, frame images that are temporally adjacent are
Since it does not change so much, the frame image P
When the difference (PC2-PC1) between C1 and PC2 is calculated,
The signal of the difference image PC12 obtained as a result has a small value. Therefore, by encoding the difference image PC12 instead of the frame image PC2, the frame image PC2
The generated code amount (generated bit amount) can be reduced as compared with the case of encoding itself.

Further, in this case, the compression ratio can be further improved by performing the motion compensation on the frame image PC1 and then calculating the difference image PC12.

When performing motion compensation, first, the frame image PC2 is divided into blocks (so-called macroblocks or blocks) of a predetermined size such as 16 × 16 dots or 8 × 8 dots. Then, each of the blocks forming the frame image PC2 and the frame image PC1 are pattern-matched, and as a result, the image portion (image) on the frame image PC1 that is most similar to the luminance signal pattern in each block of the frame image PC2. Block) is detected.

The frame image P detected in this way
Offset between the position of the image portion on C1 and the position of the corresponding block of the frame image PC2 (the frame image PC1 relative to the position of the block of the frame image PC2
The position of the upper image portion) is detected as a motion vector.

If the frame image PC1 is motion-compensated corresponding to the motion vector, that is, the frame image is moved by the motion vector, the frame image PC1 itself is more than the frame image PC2 if the focus is on the block to be encoded. , The difference image P of the frame image PC2 with respect to the motion-compensated frame image PC1.
In C12, the signal value becomes smaller than that in the case where motion compensation is not performed, so that the generated code amount can be further reduced.

Here, hereinafter, the frame image PC2 is appropriately used.
Like the reference image or the frame image PC1,
An image subtracted from the reference image is called a search image.

The motion vector calculated as above and the difference image PC12 are usually variable-length coded and transmitted.

By the way, by pattern matching,
When searching for an image portion on the search image (frame image PC1) that is most similar to the pattern of the luminance signal of the block of the reference image (frame image PC2), for example, it is performed as follows.

That is, for example, as shown in FIG. 9, the block on the reference image which is the current encoding target is set to A, and on the search image at the position deviated from the position of this block A by the vector (ΔX, ΔY). , The pixel value A (X, Y) belonging to the block A (where (X, Y) is the pixel coordinate) and the pixel value A '(X + ΔX, Y + ΔY) belonging to the block A ′. ) And the absolute value sum E (ΔX, ΔY) of the difference are calculated according to the following equation.

E (ΔX, ΔY) = Σ | A (X, Y) −A ′ (X + ΔX, Y + ΔY) | (1) where Σ is the entire coordinates (X, Y) included in the block A. Means a summation about.

Here, E (ΔX, ΔY) has a smaller value as the images in the blocks A and A ′ are closer to each other from the equation (1). Therefore, E (ΔX, ΔY)
Corresponds to an error (residual error) between blocks A and A ′, so to speak, and is hereinafter referred to as a residual error (or error).
It should be noted that, as described above, this residual error is caused by blocks A and A ′.
Since it is the sum of the absolute values of the differences with, it also corresponds to the code amount of the difference image.

According to a search algorithm such as so-called full search or multi-step search, all vectors (ΔX, ΔY) in the search range (since these are candidates for motion vectors, they will be referred to as candidate vectors hereinafter). On the other hand, the residual E (ΔX, ΔY) is calculated, and the vector (ΔX, ΔY) that minimizes the value is detected as the motion vector described above.

[0017]

By the way, in order to obtain a motion vector according to the actual motion of an image, it is necessary to be able to deal with a fast (large) moving object on the image, for example. And therefore the search range,
That is, it is necessary to widen the range designated by the candidate vector.

That is, for example, as shown in FIG.
If (ΔX, ΔY) has a distribution as shown by contour lines in the figure, and the position indicated by ● is the position that minimizes the residual E (ΔX, ΔY), the search range Is a narrow range with a dark shadow, the residual E (Δ
A vector Vb indicated by a dotted arrow in the figure that minimizes (X, ΔY) is detected as a motion vector. Therefore, in this case, a motion vector that accurately reflects the actual motion of the image cannot be obtained.

On the other hand, when the search range is a wide range with a light shadow, the vector Va indicated by the thick solid arrow in the drawing that moves the residual E (ΔX, ΔY) to the true minimum is moved. Detected as a vector. So in this case,
It is possible to obtain a motion vector that matches the actual motion of the image.

When the search range is widened in this way, it becomes possible to cope with fast movements as described above. However, for example, if the motion of the object on the image is relatively small and its shape is flat, or if the image is a repetitive texture, then simply (true) the minimum residual E (ΔX, Δ) is obtained. The candidate vector that gives Y) may have a very large length.
Therefore, in this case, the candidate vector of that large length is
Even though the image does not actually have that much motion, it is used as a motion vector. That is, in this case, a so-called false motion vector that does not match the actual motion of the image is detected.

Further, as described above, when the motion vector is variable-length coded, if the size thereof is long, the generated code amount increases. Therefore, in this case, even if the generated code amount for the difference image is small due to the generation of the motion vector having a large length, a combination of the generated code amount of the motion vector and the generated difference image is encoded without motion compensation. Than that.

This is not only the case where the movement of the object on the image is relatively small and its shape is flat, or the image has a repetitive texture. In a certain case, if the search range is wide, a motion vector that matches the actual motion of the image can be obtained as described above. Therefore, the generated code amount of the difference image decreases, but the motion vector is Since the movement of the object is large, as shown in FIG. 11, the length thereof becomes large, which causes a problem of increasing the total generated code amount.

That is, as shown in FIG. 12, it is preferable to use a vector Vd having a short size, which represents the motion of an image with a certain degree of accuracy (accurately) as the motion vector, without minimizing the residual error. Although the generated code amount can be reduced, according to the conventional method, since the vector Vc that minimizes the residual is simply used as the motion vector, the generated code amount increases as a whole. There were challenges.

Therefore, in coding the motion vector, there is a method of coding the difference between the motion vector of the neighboring block and the like. In this method, if the motion of the image is almost uniform, the generated code amount is Can be reduced,
When the change in the movement of the image is large, the generated code amount still increases.

Further, as shown in FIG. 13, a method for predicting the motion of an object called a telescopic search in which the motion vector generated in the immediately preceding frame image is referred to and the current search range is limited to the vicinity of the motion vector. It has been known.

However, in the telescopic search, since the search range is limited, when the prediction is incorrect, that is, when there is a sudden movement in the image that is different from the past, as shown in FIG. It is difficult to detect a motion vector that matches the motion. Even so, according to the telescopic search, the candidate vector that minimizes the residual error is detected as the motion vector within the current search range. Therefore, for example, as shown in FIG. 14, when the true motion direction is opposite to the search range direction, even if a difference image is generated by performing motion compensation using the detected motion vector, the signal The value will not be small, and as a result
There is a problem that the amount of generated codes increases.

On the other hand, as a method for preventing an increase in the generated code amount due to the detection of the false motion vector described above, a candidate vector having a short size is preferentially selected as a motion vector, for example, as shown in FIG. As shown in FIG. 15 (b), weighting is performed such that the larger the motion is, the larger the residual is. (For example, the weighting function W (ΔX)
(Multiplying) is possible. In FIG. 15, only the ΔX direction is considered and the ΔY direction is ignored.

In the conventional case, for example, for a flat texture image having random noise, for example, as shown in FIG.
The residual E (ΔX) as shown in (a) (however, E (ΔX)
Is obtained when E (ΔX, ΔY) in the ΔY direction is ignored), the point X _{0 which} is the minimum value and is detected as the motion vector is detected as the motion vector. However, according to the case of weighting, the residual E (Δ
X) is the weighting function W (ΔX) shown in FIG.
E shown in FIG. 15C, which is obtained by multiplying by
The point X ₁ that gives the minimum value of (ΔX) · W (ΔX) and is close to the origin is detected as a motion vector.

However, according to this method, a motion vector that emphasizes the background in the image is detected, and when there is an object moving on the image with respect to the background, a false motion vector is also detected. There were challenges.

The present invention has been made in view of such a situation, and it is possible to prevent a false motion vector from being detected and to reduce the generated code amount.

[0031]

According to a first aspect of the present invention, an image encoding method detects a motion vector that minimizes a predetermined error from an image signal by a pattern matching method,
A predicted image signal is obtained by performing motion compensation based on a motion vector, a difference between the image signal and the predicted image signal is calculated, and the difference data and the motion vector are encoded by a variable length image coding method, It is characterized in that a predetermined error is weighted by 2 or more based on the motion vector detected in the past.

In the image coding method according to the second aspect, a motion vector that minimizes a predetermined error is detected from the image signal by a pattern matching method, and motion compensation is performed based on the motion vector to predict a predicted image. Ask for a signal,
In an image coding method of calculating a difference between an image signal and a predicted image signal and performing variable length coding of the difference data and a motion vector, the size of the motion vector is reduced and variable with respect to a predetermined error. A feature is that weighting of 2 or more is performed so as to reduce the amount of bits generated by long coding.

In the image coding method according to the third aspect, a motion vector is detected for each predetermined block of an image of one frame, and one of two or more weightings is a motion of a block of an image of the same frame. It is characterized in that it corresponds to a motion vector already detected among the vectors.

The image coding method according to claim 4 is characterized in that one of the two or more weightings corresponds to the already detected motion vector of the image of the already coded frame. To do.

[0035]

In the image coding method according to claim 1,
The predetermined error is weighted by 2 or more based on the motion vector detected in the past, and the motion vector that minimizes the predetermined error obtained as a result is detected. Therefore, it is possible to detect an almost accurate motion vector,
Moreover, the amount of generated codes can be reduced.

In the image coding method according to the second aspect, weighting of 2 or more is performed so as to reduce the magnitude of the motion vector with respect to a predetermined error and reduce the bit generation amount due to the variable length coding. And a motion vector that minimizes the resulting predetermined error is detected. Therefore, it is possible to detect an almost accurate motion vector and reduce the generated code amount.

In the image coding method of the third aspect, one of the two or more weightings corresponds to the already detected motion vector of the motion vectors of the blocks of the image of the same frame. Therefore, it is possible to obtain a motion vector that accurately reflects the motion of the background of the image.

In the image coding method according to the fourth aspect, one of the two or more weightings corresponds to the already detected motion vector of the image of the already coded frame, so that the image It is possible to obtain a motion vector that accurately reflects the motion of the object inside.

[0039]

1 is a block diagram showing the configuration of an embodiment of an image coding apparatus to which the image coding method of the present invention is applied. In this image coding apparatus, a signal processing circuit (not shown) supplies, for example, an image that is divided into blocks (macroblocks) such as 16 × 16 in units of frames. An image to be coded out of the blocked images is input to the current memory 1 as a reference image and temporarily stored therein. Then, a block of a reference image (hereinafter, appropriately referred to as a reference image block) is read from the current memory 1 in the order from left to right and from top to bottom in the same manner as the scanning direction of the image, and the difference unit 3 And is supplied to the calculator 15.

The search memory 2 stores the residual E with respect to the reference image.
The one for which (ΔX, ΔY) is calculated (for example, the image one frame before or one frame after the reference image) is input as the search image, and the search memory 2 is The search image is temporarily stored. Further, the search memory 2 is sequentially supplied with candidate vectors (ΔX, ΔY) generated by a vector search circuit (not shown) by a full search method or the like.
That is, the search memory 2 is supplied with candidate vectors (ΔX, ΔY) that sequentially indicate a predetermined wide search range.

From the search memory 2, the image block indicated by the candidate vector (ΔX, ΔY) in the search image is read out and supplied to the difference unit 3. Therefore, the image blocks within the search range (hereinafter, appropriately referred to as a search image block) are sequentially read from the search memory 2 according to the candidate vector (ΔX, ΔY) and supplied to the difference unit 3. become.

The differencer 3 takes the difference between the pixel included in the reference image block from the current memory 1 and the pixel included in the search image block from the search memory 2, and further calculates the absolute value thereof, for example. That is, the pixel at the coordinate (X, Y) of the reference image block is set to A (X, Y), and the pixel of the search image block separated from the reference image block by the candidate vector (ΔX, ΔY) is set. , A ′ (X + ΔX, Y + ΔY), in the subtractor 3, the equation S (X, Y) = | A (X, Y) −A ′ (X + ΔX, Y +
According to ΔY) |, the absolute value S (X, Y) of the difference between the pixel of the reference image block and the pixel of the search image block is calculated on a pixel-by-pixel basis and sequentially output to the cumulative adder 4. The difference unit 3 may calculate the square value of the difference in addition to the absolute value of the difference between pixels.

The cumulative adder 4 cumulatively adds S (X, Y) of one pixel unit output from the differentiator 3 in one block unit of the reference image, whereby the above-described residual E (ΔX, △
Y) is calculated.

Therefore, the differentiator 3 and the cumulative adder 4 perform the operation represented by the above-mentioned equation (1), whereby the candidate vectors (ΔX, Δ) within the search range are calculated.
The residual E (ΔX, ΔY) for each Y) is obtained.

The cumulative adder 4 calculates the absolute value S of the difference between the pixels.
When the cumulative addition value of 1 block of the reference image of (X, Y), that is, the residual E (ΔX, ΔY) is calculated, it is calculated by the calculator 5
To output sequentially.

In addition to the residual E (ΔX, ΔY), the calculator 5 has a weight W1 (Δ) obtained from the weight calculator 13 or 14 based on the motion vector detected in the past.
X, ΔY) or W2 (ΔX, ΔY) is also supplied (details will be described later).

The arithmetic unit 5 calculates the residual E (ΔX, ΔY) obtained by the pattern matching method as described above.
For example, the weight W1 (ΔX, ΔY) or W2 (ΔX, ΔY) supplied from the weight calculator 13 or 14 is multiplied, whereby the residual E (ΔX, ΔY) is weighted. W
A weighted residual (hereinafter, appropriately referred to as a weighted residual) F (ΔX, ΔY) corresponding to 1 (ΔX, ΔY) and W2 (ΔX, ΔY) is calculated. This weighted residual F
(ΔX, ΔY) is supplied to the minimum residual detection circuit (Min) 6.

The minimum value detection circuit 6 detects the minimum of the weighted residuals F (ΔX, ΔY) for each of the candidate vectors (ΔX, ΔY) within the search range,
A candidate vector (ΔX, ΔY) that gives the minimum weighted residual F (ΔX, ΔY) is a motion vector (ΔX, ΔY).
It is designed to be output as. The motion vector (ΔX, ΔY) is supplied to the variable length coding circuit 7, the motion compensation circuit (MC) 11, and the previous / current frame vector storage memory 12.

In the variable length coding circuit 7, the motion vector from the minimum value detection circuit 6 is the motion vector (minimum value detection circuit 6) supplied immediately before from the minimum value detection circuit 6 as well.
From the block immediately preceding the block corresponding to the motion vector just supplied (for example, the motion vector of the block adjacent to the left side), the difference is calculated according to a predetermined variable length coding table. It is long-coded and supplied to a multiplexer (MIX) 9. Therefore, when the motion vector matches the motion vector of the block on the left, the coding efficiency can be improved. When there is no adjacent block on the left side, that is, when the block to be encoded is the leftmost block of the frame, for example, the motion vector of the block is directly variable-length encoded.

On the other hand, in the motion compensation circuit 11, the image read from the local decode memory 10 is subjected to motion compensation corresponding to the motion vector from the minimum value detection circuit 6, and is supplied to the arithmetic unit 15. The search image stored in the search memory 2 is already encoded in the local decode memory 10 and decoded by a local decoder (not shown) is supplied and stored as a predicted original image. .

Therefore, from the motion compensation circuit 11, the prediction original image (search image) that minimizes the weighted residual F (ΔX, ΔY) with respect to the block of the reference image that is currently to be encoded. The motion compensation is applied to the image portion, and the operation unit 15
Will be output to.

The arithmetic unit 15 obtains the difference between the reference image block supplied from the current memory 1 and the image block supplied from the motion compensation circuit 11, and the difference image is supplied to the variable length coding circuit 8. To be done. In the variable length coding circuit 8, the difference image from the arithmetic unit 15 is variable length coded according to a predetermined variable length coding table and output to the multiplexer 9. The variable length coding circuit 8
In the above method, the difference image is subjected to orthogonal transform processing such as DCT processing, if necessary, and the resulting orthogonal transform data is quantized in a predetermined quantization step and variable length coded. be able to.

In the multiplexer 9, the variable-length coded data from the variable-length coding circuits 8 and 9 (variable-length coded difference image and variable-length coded difference of motion vector as described above). ), And other necessary information are multiplexed and transmitted to the receiving side via a transmission path (not shown), or recorded on a recording medium (e.g., magneto-optical disk or magnetic tape) not shown.

On the other hand, the motion vector (ΔX, ΔY) supplied from the minimum value detection circuit 6 to the storage memory 12 is temporarily stored therein. The storage memory 12 can store the motion vector of at least one frame image.

The storage memory 12 performs the current encoding based on the motion vector of the image of the frame encoded before the frame of the reference image which is the current encoding target among the stored motion vectors. The predicted value (estimated value) of the motion vector of the reference image block to be
First weight center vector) V1 (= Vx, Vy)
And outputs it to the weight calculator 13.

Here, what is called telescopic prediction is used as the first weight center vector V1 (this is the object image in the reference image block (there is an image serving as a background, and moving on the background image). Image of the object) is a vector that indicates the position of the center of the range where there is likely to be a motion vector that reflects the motion). That is, as the first weight center vector V1, for example, as shown in FIG.
As shown in, a frame (hereinafter, referred to as a current frame) coded immediately before a frame of the reference image (hereinafter, referred to as a current frame).
The motion vector of the block at the same position as the reference image block of the image (search image) of the previous frame is used (but is not limited to this). FIG. 2 shows a case in which a frame that is one frame before the current frame in time is the previous frame.

In addition to the first weight center vector V1 (= (Vx, Vy)), the weight calculator 13 also calculates candidate vectors (Δ).
X, ΔY) is also provided, where the residual E corresponding to the candidate vector (ΔX, ΔY) is obtained, for example, according to the first weighting function W1 given by the following equation.
The first weight W1 (ΔX, ΔY) added to (ΔX, ΔY)
Y) is calculated. W1 (ΔX, ΔY) = K ₁ × √ ((ΔX-Vx) ² + (ΔY-Vy) ² ) (2) where K ₁ is a predetermined proportional constant.

The first weight W1 (ΔX, ΔY) is
It can be obtained by a function other than the above equation. In addition, this function is obtained by adding the residual E (ΔX, ΔY) corresponding to the candidate vector (ΔX, ΔY) to the argument (W
1 (ΔX, ΔY, E (ΔX, ΔY)) (however, in this case, as shown by the dotted line in FIG. 1, the residual E (ΔX, ΔY)). Y) must be supplied from the cumulative adder 4 to the weight calculator 13).

That is, the function for giving the first weight W1 is determined by, for example, a predetermined variable length coding table used by the variable length coding circuit 7 or 8 for variable length coding the motion vector difference or the difference image. You can change it. However, this function gives the minimum first weight W1 when the candidate vector (ΔX, ΔY) matches the first weight center vector V1, as shown in FIG. [Delta] X, [Delta] Y) increases with increasing distance from the first weight center vector V1.
It must be a monotonically increasing function that gives 1.

Here, in FIG. 3, the first weight W1
The distribution state of is represented by contour lines, but the value (weight value) of the first weight W1 attached to this contour line is not absolute, and for example, the minimum value of the first weight W1 is 0. It is a relative value when expressed.

Returning to FIG. 1, the storage memory 12 stores the first weighted center vector V1 and the coded motion vector stored before the reference image block to be coded. A predicted value (estimated value) of the motion vector of the reference image block that is about to be encoded (hereinafter, referred to as a second weight center vector) V2 (= Zx, Z) based on the motion vector of the encoded reference image block.
y) is calculated and output to the weight calculator 14.

That is, as shown in, for example, FIG. 4, the storage memory 12 stores the already-encoded block a adjacent to the block e among the blocks in the vicinity of the block e which is the current frame to be encoded. Through d (however, blocks a to d are adjacent to the upper left, right above, upper right, and left side of the block e, respectively). A vector in which the difference between the motion vectors that are variable-length coded by the coding circuit 7 becomes small, and as a result, the code amount (bit amount) of the variable-length coded data output from the variable length coding circuit 7 decreases. , For example, the average value of the motion vectors a to d, and the second weighted center vector V2 (this is a motion vector reflecting the motion of the background image in the reference image block). And as a vector) for instructing the center and a position of the likely range, and outputs to the weight calculator 14.

Here, as the second weight center vector V2, it is possible to use only the motion vector of the one block on the left adjacent to which the difference is taken when variable length coding is performed as described above. However, in this case, it may not be possible to accurately predict the motion in the block that is currently the encoding target, so as described above,
It is preferable that the motion vectors of the blocks in the vicinity of the block to be coded are comprehensively taken into account by a method such as averaging the motion vectors.

When the average value of the motion vectors of the blocks in the vicinity of the block to be coded is the second weighting center vector V2, this second weighting center vector V
2 is usually a vector near the origin (0, 0) unless a large-length motion vector is generated in a nearby block.

In addition to the second weight center vector V2 (= (Zx, Zy)), the weight calculator 14 also calculates the candidate vector (Δ).
X, ΔY) is also supplied, where the residual E corresponding to the candidate vector (ΔX, ΔY) is calculated according to the second weighting function W2 given by the following equation, for example.
The second weight W2 (ΔX, ΔY) added to (ΔX, ΔY)
Y) is calculated. W2 (ΔX, ΔY) = K ₂ × √ ((ΔX−Zx) ² + (ΔY−Zy) ² ) ... (3) where K ₂ is a predetermined proportional constant.

The second weight W2 (ΔX, ΔY) is
It can be obtained by a function other than the above equation. In addition, this function is obtained by adding the residual E (ΔX, ΔY) corresponding to the candidate vector (ΔX, ΔY) to the argument (W
2 (ΔX, ΔY, E (ΔX, ΔY)) (however, in this case, as shown by the dotted line in FIG. 1, the residual E (ΔX, ΔY)) Y) needs to be supplied from the cumulative adder 4 to the weight calculator 14).

That is, the function for giving the second weight W2 is the same as the function for giving the first weight W1 described above, for example, the variable length coding circuit 7 for variable length coding the difference of the motion vector or the difference image or 8 can be changed according to a predetermined variable length coding table used by each. However, as shown in FIG. 5 corresponding to FIG. 3, this function has the minimum second weight W2 when the candidate vector (ΔX, ΔY) matches the second weight center vector V2.
And the candidate vector (ΔX, ΔY) needs to be a monotonically increasing function that gives a larger value of the second weight W2 as the candidate vector (ΔX, ΔY) moves away from the second weight center vector V2.

Here, also in FIG. 5, as in the case of FIG. 3, the distribution state of the second weight W2 is shown by contour lines, but the value of the second weight W2 attached to this contour line ( The weight value) is not an absolute value, but is a relative value when the minimum value of the second weight W2 is represented as 0, for example.

As described above, the first weight W1 (ΔX, Δ) calculated by the weight calculator 13 or 14 respectively.
Y) or the second weight W2 (ΔX, ΔY) is calculated by the calculator 5
Is supplied to. In the calculator 5, as described above, the residual E
(ΔX, ΔY) is multiplied by the first weights W1 (ΔX, ΔY) and W2 (ΔX, ΔY), that is, the formula F (ΔX, ΔY) = E (ΔX, ΔY) × W1 (ΔX, ΔY) × W2 (ΔX, ΔY) (4) The weighted residual F (ΔX, ΔY) is calculated and the minimum value detection circuit is calculated. 6 is output.

In the minimum value detection circuit 6, as described above, the candidate vector (Δ) within the search range is also included.
Weighted residual F (ΔX, ΔY) for each of X, ΔY)
The smallest one of Y) is detected, and the candidate vector (ΔX, ΔY) giving the smallest weighted residual F (ΔX, ΔY) is detected.
ΔY) is output as a motion vector (ΔX, ΔY).

Here, FIG. 6 shows the first weight W1 (ΔX, ΔY) or W2 (Δ) shown in FIG. 3 or FIG. 5, respectively.
(X, ΔY) is shown. From FIG. 6, the first or second weight center vector V1 or V2
The weighted residual F (ΔX, ΔY) for the candidate vector (ΔX, ΔY) in the vicinity of is relatively small, and is a candidate vector distant from the first or second weight center vector V1 or V2. Weighted residual F for (ΔX, ΔY)
It can be seen that (ΔX, ΔY) becomes relatively large.

As described above, the first weighting center vector V1 indicates the position at the center of the range in which the motion vector reflecting the motion of the object image in the reference image block is likely to exist, and the second weighting center vector V1. The weighting center vector V2 of (1) indicates the center position (usually near the origin as described above) of the range in which the motion vector reflecting the motion of the background image in the reference image block is likely to exist. By setting the candidate vector (ΔX, ΔY) giving the minimum value of the weighted residual F (ΔX, ΔY) obtained according to the equation (4) as the motion vector (ΔX, ΔY), The motion vector has a short size and reflects the motion of both the background and the object in the reference image. In this way, since the motion vector reflects the motion of the image, it is possible to reduce the code amount of the difference image. Further, since the motion vector has a short size, the code amount of the entire image is reduced. Can be reduced.

Further, it is possible to reduce the amount of generated code when the motion vector is directly variable-length coded.

Further, FIG. 7A shows the first weight W1.
Considering only the ΔX direction of the weights multiplied by (ΔX, ΔY) or W2 (ΔX, ΔY) and ignoring the ΔY direction W1 (ΔX) × W2 (ΔX) Shows.

This is obtained from, for example, an image of a flat texture with random noise, as shown in FIG.
The weighted residual, that is, the weighted residual shown in FIG. 7B is obtained by multiplying the residual E (ΔX) shown in (a).
It becomes something like. Therefore, in this case, in the figure, X
The first or second weight center vector V indicated by ₂ or X _3.
It becomes easy to detect the vicinity of the position indicated by 1 or V2 as a motion vector, and it is possible to prevent detection of a false motion vector that does not match the actual motion of the image.

At the start of encoding, since the motion vector is not yet stored in the storage memory 12 (FIG. 1), the calculator 5 causes the residual error E from the cumulative adder 4 to be calculated.
(ΔX, ΔY) is directly output to the minimum value detection circuit 6. Therefore, in this case, the motion vector is detected as in the conventional device.

In the present embodiment, the residual E (ΔX, Δ
Y) two weights W1 (ΔX, ΔY) and W2
Weighting is performed by multiplying (ΔX, ΔY), but in addition to this, for example, two weights W1 (ΔX, ΔY) and W2 are applied to the residual E (ΔX, ΔY). (△ X,
Weighting can be performed by adding ΔY). However, in this case, it is necessary for the computing unit 5 to perform addition processing instead of multiplication processing.

Further, in this embodiment, the residual E (Δ
Two weights W1 (ΔX, ΔY) and W2 (ΔX, ΔY) (or W1 (ΔX, ΔY, E) for X, ΔY)
(ΔX, ΔY)) and W2 (ΔX, ΔY, E (ΔX,
ΔY))) is used for weighting. In addition to this, other weights based on motion vectors detected in the past (the size of the motion vector is reduced, and
It is possible to perform weighting of 3 or more by using a weight for reducing the generated code amount by the variable length coding.

That is, for example, the weighted residual F (ΔX, ΔY) can be calculated according to the following equation.

F (ΔX, ΔY) = E (ΔX, ΔY) × W1 (ΔX, ΔY) × W2 (ΔX, ΔY)

F (ΔX, ΔY) = E (ΔX, ΔY) × W1 (ΔX, ΔY, E (ΔX, ΔY)) × W2 (ΔX, ΔY, E (ΔX, ΔY))

F (ΔX, ΔY) = E (ΔX, ΔY) + W1 (ΔX, ΔY) + W2 (ΔX, ΔY)

F (ΔX, ΔY) = E (ΔX, ΔY) + W1 (ΔX, ΔY, E (ΔX, ΔY)) + W2 (ΔX, ΔY, E (ΔX, ΔY))

Further, in the present embodiment, the processing is performed on the image in frame units, but the same processing can be performed on the image in field units.

Further, in the present embodiment, the weighting ratio by the first and second weights W1 and W2 was not mentioned, but the variable length coding circuit 7 calculates the difference from the motion vector of the adjacent block. Considering variable length coding, the generated code amount is further reduced by increasing the influence of the weighting by the second weight W2 over the first weight W1 (widening the dynamic range of W2 over W1). be able to. However, for an image in which a small object is moving on a background that is panning, for example, it is preferable to make the weighting effect of the first weight W1 stronger than the second weight W2.

The present invention is also applicable to, for example, MPEG or JPE.
It can be applied to an image encoding device conforming to G or the like.

[0087]

According to the image coding method of the first aspect, the predetermined error is weighted by 2 or more based on the motion vector detected in the past, and the predetermined error obtained as a result is obtained. Since the motion vector that minimizes the error is detected, the generated code amount can be reduced.

According to the image coding method of the second aspect, the magnitude of the motion vector is reduced with respect to a predetermined error, and the bit generation amount due to the variable length coding is reduced to 2 or more. Since the weighting is performed and the motion vector that minimizes the predetermined error obtained as a result is detected, the generated code amount can be reduced.

According to the image coding method of the third aspect, one of the two or more weightings corresponds to the already detected motion vector of the motion vectors of the blocks of the image of the same frame. Therefore, it is possible to obtain a motion vector that accurately reflects the motion of the background of the image.

According to the image coding method of the fourth aspect, since one of the two or more weightings corresponds to the already detected motion vector of the image of the already coded frame, It is possible to obtain a motion vector that accurately reflects the motion of the object in the image.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of an embodiment of an image encoding device to which an image encoding method of the present invention is applied.

FIG. 2 is a diagram illustrating a first weight center vector V1 output from a storage memory 12.

FIG. 3 is a diagram illustrating a first weight W1 calculated by a weight calculator 13.

FIG. 4 is a diagram illustrating a second weight center vector V2 output from the storage memory 12.

FIG. 5 is a diagram illustrating a second weight W2 calculated by a weight calculator 14.

6 is a diagram showing weights obtained by multiplying the first or second weights W1 or W2 shown in FIG. 3 or FIG. 5, respectively.

FIG. 7 is a diagram showing the weights shown in FIG. 6 and residuals weighted by the weights.

FIG. 8 is a diagram illustrating the principle of image compression using inter-frame correlation.

FIG. 9 is a diagram illustrating the principle of motion vector detection.

FIG. 10 is a diagram showing how a motion vector is detected when the search range is wide and narrow.

[Fig. 11] Fig. 11 is a diagram illustrating a motion vector having a large amount of generated code after variable-length coding.

FIG. 12 is a diagram for explaining the relationship between the amount of generated code after variable-length coding and a motion vector.

FIG. 13 is a diagram illustrating a telescopic search.

FIG. 14 is a diagram for explaining a situation in the telescopic search when the prediction of the motion of the image is incorrect.

FIG. 15 is a diagram illustrating a conventional weighting method.

[Explanation of symbols]

 1 Current Memory 2 Search Memory 3 Difference Unit 4 Cumulative Adder 5 Computing Unit 6 Minimum Value Detection Circuit 7, 8 Variable Length Coding Circuit 9 Multiplexer 10 Local Decode Memory 11 Motion Compensation Circuit 12 Previous / Current Frame Vector Accumulation Memory 13, 14 Weight calculator 15 calculator

Claims (4)

[Claims] 1. A pattern matching method is used to detect a motion vector that minimizes a predetermined error from an image signal, and motion compensation is performed based on the motion vector to obtain a predicted image signal. An image encoding method for calculating a difference between an image signal and variable length encoding the difference data and the motion vector, wherein the difference is 2 or more based on a motion vector detected in the past with respect to the predetermined error. An image coding method characterized by performing weighting of the. 2. A pattern matching method is used to detect a motion vector that minimizes a predetermined error from the image signal, and motion compensation is performed based on the motion vector to obtain a predicted image signal. An image coding method for calculating a difference between an image signal and variable length coding the difference data and the motion vector, wherein the size of the motion vector is reduced with respect to the predetermined error, and the variable An image coding method, wherein weighting of 2 or more is performed so as to reduce the amount of bits generated by long coding. 3. The motion vector is detected for each predetermined block of an image of one frame, and one of the two or more weightings has already been detected among motion vectors of blocks of an image of the same frame. The image coding method according to claim 1, wherein the image coding method corresponds to a motion vector. 4. The method according to claim 1, wherein one of the two or more weightings corresponds to an already detected motion vector of an image of an already encoded frame. Image coding method.