JPH0832035B2 - Image coding method - Google Patents

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image coding method for performing highly efficient coding of image signals.

2. Description of the Related Art The transmission speed of a digitized image signal reaches 100 Mbps or more, and it is difficult to transmit this signal as it is using an existing communication line in terms of transmission capacity and transmission cost. High-efficiency coding is a technique for removing redundancy of an image signal and reducing transmission speed, and various types of image coding methods and devices have been conventionally proposed. Among them, the motion compensation interframe coding method and the orthogonal transform coding method, which are generally used, will be described as an example.

FIG. 5 is an example of a conventional interframe coding method,
It is a block diagram of the encoding apparatus by motion compensation inter-frame encoding. In FIG. 5, 51 is an adder, 52 is a quantizer, 53 is a predictor, 54 is a delay circuit, 55 is a motion detection circuit, and 56
Is an inverse quantizer and 57 is an adder.

The operation of the motion-compensated interframe coding apparatus configured as described above will be described below.

In FIG. 5, the input image data is added by the adder 51.
Then, the current image data and the prediction value output from the predictor 53 are subtracted to form a prediction error signal value. The prediction error signal value is quantized by the quantizer 52 and output as a coded output, and at the same time, sent to the dequantizer 56 and dequantized. The inverse quantized output is added by the adder 57 to the prediction output of the predictor 53 to form a locally decoded signal. The locally decoded signal is sent to the delay circuit 54,
One frame delay is applied. The output of the delay circuit 54 is input to the predictor 53, and the predicted value of the next frame is generated.
On the other hand, the input image data is sent to the adder 51 and the motion detection circuit 55 at the same time. The motion detection circuit 55 compares the current image data with the image data of one frame before, which is the output of the delay circuit 54, in a block of a predetermined size of m × n, and selects the block having the highest correlation. ,
At the same time, the motion vector is output. Generally, in a moving image with little motion, the correlation between pixels between frames is high, and the variance value of the prediction error signal is smaller than the variance value of the input signal. Therefore, a high compression rate can be realized by encoding the inter-frame prediction signal. On the other hand, for images with large movements, there is a drawback that the efficiency decreases because the correlation between frames becomes small. However, by performing motion compensation between frames based on the motion detection,
A high compression efficiency can be obtained because the correlation between frames is maintained even in an image that moves rapidly.

Further, FIG. 6 shows an example of conventional block coding.
It is a block diagram of a three-dimensional orthogonal transformation coding device. In FIG. 6, 61 is a three-dimensional blocking circuit, 62 is an orthogonal transform circuit, and 63 is a quantizer.

The operation of the three-dimensional orthogonal transform coding device configured as described above will be described below.

In FIG. 6, the image input data input to the three-dimensional block forming circuit 61 is 3 including horizontal, vertical, and temporal directions.
It is made into a dimensional block and orthogonally transformed by the orthogonal transformation circuit 62. The transform coefficient output from the orthogonal transform circuit 62 is quantized by the quantizer 63 so that the low-sequential coefficient has many quantized bit allocations and the high-sequential coefficient has less quantized bit allocations. The quantized transform coefficient is output as an encoded output. Generally, a natural image has a high correlation between pixels in the horizontal, vertical, and temporal directions, and the orthogonal transform coefficient has a large amount of energy in a low sequence and a small amount in a high sequence. Therefore, when quantizing, even if a small number of bits are assigned to the high-sequency coefficient as described above, visual deterioration is not noticeable, and compression with suppressed image quality deterioration is possible (for example, published by Nikkan Kogyo Shimbun, Nobuhiko Fukibuki "Digital Signal Processing of Images" Chapter 9, or published by prentice-hall, Ja-ya
nt et al., "digital coding of waveform" Chapter 12).

DISCLOSURE OF THE INVENTION Problems to be Solved by the Invention However, in the configuration using inter-frame correlation as shown in FIG. 5, the encoded signal has continuity in the time direction, and it is difficult to decode from the middle of the moving image. Further, in the structure including three dimensions in the horizontal, vertical and time directions as shown in FIG. 6, in the case of an image having a large motion, there is a problem that the correlation in the time direction is lowered and the efficiency is lowered.

In view of the above problems, the present invention provides an image coding method that can easily perform decoding from the middle of a moving image and that can achieve a high compression rate even for a large moving image.

Means for Solving the Problems In order to solve the above-mentioned problems, the image coding method of the present invention is such that the input T frame image data s (h, v, t)
In the first frame, that is, s (h, v, 1), a two-dimensional block sb (x, x) of m (horizontal direction) × n (vertical direction) pixels.
y, 1), and on the adjacent second frame s (h, v, 2), a two-dimensional block sb (mb) of size mxn that has the most correlation with the sb (x, y, 1). x, y, 2), and at the same time, a horizontal motion vector mvh (x, y, 2) indicating the amount of horizontal motion and a vertical motion vector mvv (x, y, 2) indicating the amount of vertical motion. On the next adjacent third frame s (h, v, 3), the s
Two-dimensional block sb (x, y, 3) of size m × n that is most correlated with b (x, y, 2) and motion vector mvh (x, y, 3), mvv
(X, y, 3) is obtained, and motion detection is performed up to T frames s (h, v, T) in the same manner, and the two-dimensional block sb (x, y, t) and motion vector mvh (x, y, t), mvv (x,
y, t) and the motion vector mvh (x, y, t), mvv (x, y,
From t), one type of representative motion vectors mvhr (t) and mvvr (t) is obtained for each frame, and the amount of movement of the representative motion vector from the first frame is mvht (t) = mvhr.
(2) + mvhr (3) + ... + mvhr (t), mvvt (t) = m
vvr (2) + mvvr (3) + ... + mvvr (t), and the t-th time
The motion amounts mvht (t) and mvvt (t) are added to the pixel index of the image data s (h, v, t) of the frame to obtain s
(H-mvht (t), v-mvvt (t), t), then mx
The block is divided into three-dimensional blocks including n × T pixels, and the three-dimensional blocks are block-coded.

Action The present invention makes it possible to decode each block in the time direction by using the block coding according to the method described above, so that the moving image can be reproduced from the middle. Further, since the motion compensation is performed, it is possible to maintain a high correlation in the time direction and realize a high compression rate even for an image with a large motion.

Embodiment An image coding method according to an embodiment of the present invention will be described below with reference to the drawings.

FIG. 1 is an explanatory diagram of an image coding method in the first embodiment of the present invention. The operation will be described below with reference to FIGS. 1 and 2.

In FIG. 1, 1 is image data of the first frame, 2
Is the image data of the second frame, 3 is the image data of the third frame, 4 is the motion vector between the first and second frames, 5 is the motion vector between the second and third frames, and 11 is the Two-dimensional block in one frame, 12
Is a two-dimensional block in the second frame, and 31 is a two-dimensional block in the third frame. Further, in FIG. 2, 21 is a part of the image data of the first frame, 22 is a part of the image data of the second frame, and 23 is a part of the image data of the third frame.

First, FIG. 2 is an explanatory view drawn as a two-dimensional plane in the horizontal and time directions of the image data of each frame in FIG. 1, and 21 in FIG. 2 indicates the image data 1 of the first frame in FIG. 22 in FIG. 2 is image data 2 of the second frame
23 in FIG. 3 corresponds to a part of the image data of the third frame. FIG. 2 (a) is an explanatory diagram of a method of constructing a three-dimensional block when motion is not detected. Input T-frame image data s (h, v, t) (h: horizontal pixel index, 1 ≦ h ≦ M, v: vertical pixel index, 1 ≦ v ≦ N, t: time Direction pixel index, 1 ≦ t ≦ T), in the first frame 21, first, m
It is divided into two-dimensional blocks 21-1 of × n, and is taken at the same two-dimensional position such as 22-1 in the second frame 2, and the two-dimensional blocks up to T frames are collected in the time direction below. It becomes a three-dimensional block of m × n × T. In FIG. 2, up to three frames in the time direction are drawn as an example. The actual blocking shall be performed for T frames in the time direction. When no motion is detected, or when there is no motion, block coding such as orthogonal transformation is performed in units of the three-dimensional block as shown in FIG. 2 (a). Next, FIG. 2 (b) shows motion vectors 24, 25 and 26 obtained by performing motion detection on the data of the first to third frames in FIG. This motion vector is obtained by the following method. First, of the input image data, a reference frame is determined. The reference frame is shown in FIG. 2 (b).
In, the image data 21 of the first frame is used. First,
The image data of the first frame is converted into an m × n two-dimensional block 21.
-1 to 21-5. Sb the two-dimensional block
(1,1,1) to sb (5,1,1) (where sb (x, y, t), x: horizontal block coordinate in the frame, 1 ≦ x
≤ M / m, y: vertical block coordinates in the frame, 1 ≤
y ≦ N / n, t: frame index, 1 ≦ t ≦ T). Here, 21-1 of the two-dimensional blocks will be described as an example. Two-dimensional block 21-1 on the first frame
Next, the second frame is subjected to block matching, which is well known in general image processing technology, to obtain the least distorted m × n two-dimensional block 22-1 ′ or sb in the second frame. (1,1,2) is required. The amount of movement in the horizontal and vertical directions at this time is defined as the motion vector 24 of the first frame, and the motion vector is a horizontal motion vector mvh (1,1,2) indicating the amount of movement in the horizontal direction, and the vertical motion. Vertical motion vector indicating amount mvv (1,1,2) (however, mvh
In (x, y, t) and mvv (x, y, t), x: horizontal block coordinate in the frame, 1 ≦ x ≦ M / m, y: vertical block coordinate in the frame, 1 ≦ y ≦ N / n, t: frame index, 1 ≦ t ≦ T). For the third frame, a similar motion vector is enhanced with reference to the m × n two-dimensional block 22-2 ′ detected in the second frame. Hereinafter, similarly, motion vectors up to t frames are obtained for all the screens. In the following embodiments, for the sake of simplicity, it is assumed that the detected motion vectors are all vertical motion amounts mvv (x, y, t) = 0. By the way, in the case of performing block coding such as orthogonal transform, as described in the conventional example, as shown in FIG.
When a three-dimensional block such as 21-1, 22-1, and 23-1 is configured, when a motion as shown in FIG. 2B is detected, the actual motion is 21-1, 22. Since it is -1 'and 23-1', the correlation in the time direction is lost and the compression efficiency is reduced. Therefore, it is conceivable to construct a three-dimensional block along the motion vectors 24 and 25 as shown in FIG. However, since the motion vector is not the same in all the two-dimensional blocks in the same frame like 24 and 26 in FIG. 2 (b), if a three-dimensional block is configured along the motion vector, the two-dimensional blocks in FIG. Between two-dimensional blocks in (c)
As shown in 22-3 'and 22-4' or 23-3 'and 23-4', holes are formed at different motion vectors. Conversely, between the two-dimensional blocks 22-4 'and 22-
Block overlap occurs at 5 ', or at 23-4' and 23-5 '. Therefore, the horizontal and vertical motion vectors of each block are in the same direction when the number is equal to or greater than a certain threshold vth,
When the magnitudes of all motion vectors in the same frame are included in a certain range vwd, the representative motion vector mvhr in the horizontal and vertical directions in the frame, respectively.
One type of (t) and mvvr (t) (t is a frame index) is obtained. Here, the vth and vwh are experimentally determined. In this example, FIG. 2 (d) replaces the motion vector 26 in FIG. 2 (b) with the motion vector of FIG. 24 defined as the representative motion vector mvhr (1). The representative motion vector mvhr (t), mvvr in the above-mentioned screen
(T) is a histogram of the motion vector of each block in the frame, and can be obtained as the one with the highest frequency in the histogram. Alternatively, when the frequency distribution is not biased and cannot be determined only by the frequency, the average value obtained by excluding the representative motion vector from the maximum value i and the minimum value j from the motion vectors of each block in the frame. And it is sufficient. The i and j numbers are excluded from the maximum and minimum values, respectively, in order to eliminate detection errors in motion detection, and the numbers of i and j may be determined experimentally in each detection method. If there is no detection error, i, j may be 0. Next, the motion amount from the first frame of the representative motion vector is calculated as mvht (t) =
mvhr (1) ＋ mvhr (2) ＋ …… ＋ mvhr (t), mvvt
(T) = mvvr (1) + mvvr (2) + ... + mvvr (t), and the movement amounts mvht (t), mvvt (m) are added to the pixel index of the image data s (h, v, t) of the t-th frame. t) is added to obtain s (h + mvht (t), v + mvvt (t), t), and the second
A three-dimensional block including m × n × T pixels is constructed in the same manner as in FIG. When the three-dimensional block is constructed in this way, as shown in FIG. 2D, since the motion vectors are all the same in the frame, there is no problem such as perforation as in FIG. 2C. Becomes FIG. 1 is a three-dimensional view of FIG. 2 (d). If the motion is only in the horizontal direction, the motion vector is a vector in the h, t plane. So, in each frame, the representative vector
Select 4 and 5, and construct a three-dimensional block using the method described above. The three-dimensional block is 11,21,
It is constructed like 31. In the above embodiments, the fact that most of the motion vectors in a frame are the same means panning of the entire screen. Therefore, if blocking is performed as shown in FIG. 1, optimal motion compensation is performed, and by coding the three-dimensional block as shown in FIG. 1, block-direction correlation is maintained and compression efficiency is improved. Can be higher.

FIG. 3 is an explanatory diagram of an image coding method showing a second embodiment of the present invention. Similar to FIG. 2, FIG. 3 also depicts the data of each frame as a two-dimensional plane in the horizontal and time directions. In FIG. 3, reference numerals 21, 22 and 23 are the same as those in the first embodiment. FIG. 2 (a) is an explanatory view showing the result of motion detection using the first frame as a two-dimensional block and the block as a reference. 22-1 'and 23-1' are the first frame. The 2D blocks of the 2nd and 3rd frames obtained by using the 2D block 21-1 as a reference are 24, 31, 33
Are the first and second frames, and 25, 32, and 34 are motion vectors between the second and third frames. FIG. 2B is an explanatory diagram showing a method of regenerating a motion vector using the method shown in the following embodiment and forming a three-dimensional block by the regenerated motion vector. Also, FIG. 3 (c) is FIG. 3 (a).
6 is an explanatory diagram showing a detection result of a motion vector different from that of FIG.
The embodiment of FIG. 2 differs from the first embodiment in that
This is a method of constructing a three-dimensional block. The motion vector detected by the same method as that of the first embodiment is not shown in FIG. 2 (a) instead of the fact that most motion vectors in the frame are directed in the same direction as in the first embodiment. As shown, it is assumed that the motion vectors gather around the two-dimensional blocks 22-2 'and 23-3' as the second and third frames progress.
In such a case, since the direction of the motion vector is different, the representative motion vector cannot be uniquely determined within the frame as in the first embodiment. Further, assuming that the vector is uniquely defined, even if a three-dimensional block is constructed using the vector, the correlation in the time direction cannot be increased. Further, if a three-dimensional block is constructed along the motion vector shown in FIG. 3 (a), the correlation in the time direction becomes the highest, but between the blocks 23-1 'and 23-2', As shown in the figure, there is a hole, that is, a portion that cannot be encoded. Such a motion vector is obtained when zooming. Therefore, in this embodiment, first, the center blocks 21-3, 22-3, 23-3, which are the centers of motion, are obtained as follows. The motion vector mvh indicating the amount of horizontal motion
The number hz (x, t) in which (x, y, t) is 0 is calculated in the frame t for the block in the vertical direction, and the number hz (x, t) is calculated.
The horizontal direction block coordinate of the central block is the horizontal direction block coordinate cx, and the number vz of the motion vector mvv (x, y, t) indicating 0 in the vertical direction is 0.
(Y, t) is calculated for the horizontal block in the frame t, and the vertical block coordinate cy having the largest number vz (y, t) is set as the vertical block coordinate of the central block. Alternatively, in the motion vector mvh (x, y, t) indicating the amount of motion in the horizontal direction, the number hcz in which 0 is continuous
(X, t) is calculated for a vertical block in the frame t, and the horizontal block coordinate cx having the largest number hcz (x, t) is set as the horizontal block coordinate of the central block, and Motion vector mv indicating the amount of motion of
Of v (x, y, t), the number vcz (y, t) in which 0 is consecutive is calculated in the frame t for horizontal blocks, and the vertical block having the largest number vcz (y, t). Coordinate cy,
The block coordinates in the vertical direction of the central block are used. The latter method is a little more complicated than the former when it is implemented as a device, but if there is an erroneous detection of a motion vector, the vector of motion amount = 0 obtained by the erroneous detection can be eliminated, so it is effective. .

Next, the representative motion vectors 35, 36, 39, 40 are obtained by the following method. Of the motion vectors in each frame, the motion vector mv of the block apart from the central block by the block coordinates x ′, x ″, excluding the block including the end points of the screen.
h (cx−x ′, y, t), mvh (cx + x ″, y, t) 1 <y <N / n
The histogram of the range is taken, and the one with the highest frequency in the histogram is the representative motion vector in the horizontal direction.
mvhr1 (t) and mvhr2 (t), and among the motion vectors in each frame, except for the block including the end point of the screen, the motion vectors mvv (x , cy−y ′, t), mvh (x, c
y + y ′, t), 1 <x <M / m, and the histogram with the highest frequency is used as the vertical representative motion vectors mvvr1 (t) and mvvr2 (t).
In this embodiment, since there is no motion in the vertical direction, only the horizontal representative motion vector is obtained. Now x ′ = x ″ = 2,
If cx = 3, mvh (3-2, y, t), mvh (3 + 2, y, t)
Histograms are calculated in the range of 1 <y <N / n. Next, the one with the highest frequency in the histogram is the representative motion vector in the vertical direction, that is, 35 in FIG. 3 (b).
Is mvhr1 (2), 39 is mvhr2 (2), 36 is mvhr1 (3), 4
0 will be obtained as mvhr2 (3). Alternatively, when the frequency distribution is not biased and cannot be determined by the frequency alone, an average value obtained by excluding i pieces from the maximum value and j pieces from the minimum value is set as the representative motion vector. The i and j numbers are excluded from the maximum and minimum values, respectively, in order to eliminate detection errors in motion detection, and the numbers of i and j may be determined experimentally in each detection method. If there is no detection error, i, j may be 0. As described above, the regenerated motion vector mvht ′ (x, y, t), mvvt ′ (x, y, t) in the horizontal and vertical directions of each block is calculated using the obtained central block and representative motion vector. Calculate from the following formula.

mvh ′ (x, y, t) = mvhr1 (t) * (cx−x) / (cx−
x ′) (where x <cx) (1) mvh ′ (x, y, t) = mvhr2 (t) * (x−cx) / (x ″ −c
x) (however, x ≧ cx) (2) mvv ′ (x, y, t) = mvrv1 (t) * (cy−y) / (cy−
y ′) (where y <cy) (3) mvv ′ (x, y, t) = mvrv2 (t) * (y−cy) / (y ″ −c
y) (however y ≧ cy) (4) where cx, cy are the block coordinates of the central block, mvhr1
(T), mvhr2 (t), mvvr1 (t), mvvr2 (t) are representative motion vectors, and x'is the representative motion vector mvhr1 (t).
In the following, the block coordinates correspond to x ″ for mvrh2 (t), y ′ for mvvr1 (t), and y ″ for mvvr2 (t). Alternatively, the following formula may be used.

mvh '(x, y, t) = mvhr1 (t) * (cx-x) ² / (cx-x') ² (where x <cx) (5) mvh '(x, y, t) = mvhr2 (t) * (x-cx) ² / (x ″ -cx) ² (where x ≧ cx) (6) mvv ′ (x, y, t) = mvvr1 (t) * (cy-y) ² / (cy-y ′) ² (where y <cy) …… (7) mvv ′ (x, y, t) ＝ mvvr2 (t) * (y-cy) ² / (y ″ -cy) ² ( However, y ≧ cy) (8) Whether to use equations (1) to (4) and equations (5) to (8) is selected so that more optimal compensation can be obtained by applying to the image. .

Next, the regenerated motion vector mvh ′ (x, y, t), mv
Using v ′ (x, y, t), the amount of motion of each block from the first frame is mvht ′ (x, y, t) = mvhr ′ (x, y, 1) + mvhr ′
(X, y, 2) ＋ …… ＋ mvhr ′ (x, y, t), mvvt ′ (x, y, t)
= Mvvr '(x, y, 1) + mvvr' (x, y, 2) + ... +, mvvr '
(X, y, t), the image data s (h, v, t) of the t-th frame
To the pixel index of mvht ′ (x, y, t), mvv
Adding t '(x, y, t), s (h + mvht' (x, y, t), v + m
vvt ′ (x, y, t), t), 2 in the t-th frame
A two-dimensional block sb ′ (x, y, t) is obtained, and the two-dimensional block sb ′ (x, y, t) is combined into a continuous T frame,
A three-dimensional block including m × n × T pixels is constructed. By doing so, there is no hole between blocks as shown in FIG. 3 (a), and the correlation in the time direction is maintained. Therefore, high compression efficiency can be realized by block-coding the three-dimensional block. Further, in the second embodiment, 21 is the image data of the first frame and 22 is the image data of the second frame. However, when the image is zoomed in, it is shown in FIG. 3 (c). As described above, a motion vector whose time relationship is completely opposite to that of FIG. 3A is detected. When such a vector is detected, the motion vector is detected again with reference to the latest frame in the time direction. That is,
If the reference frame is a T frame, T, T-1, ...
..... Motion detection is performed in the order of 3,2,1 frames. By doing so, the motion vector shown in FIG. 3 (c) is equivalently the same motion vector as in FIG. 3 (a). Therefore, effective compression can be performed by the same method as in the second embodiment.

FIG. 4 is an explanatory diagram of an image coding method showing a third embodiment of the present invention. Similar to FIGS. 2 and 3, FIG. 4 also depicts the image data of each frame as a two-dimensional plane in the horizontal and time directions. In FIG. 4, reference numerals 21, 22, 23 are the same as those in the second embodiment. It is a thing. Also, as in FIG. 3, in FIG.
It is assumed that the image data of the frame is composed of 5 blocks in the horizontal direction, and 21-1 and 21-5 are the first frame,
22-1 and 22-5 represent the blocks of the second frame, and 23-5 represents the end blocks of the image data of the third frame. Now
As shown in the first embodiment,
It is assumed that three-dimensional blocking is performed using the representative motion vectors of 41 and 42. FIG. 4 (a) is a diagram showing the result of blocking. 21-2 to 21-5 are normally blocked, but 21-1 and 22-1 cannot be blocked because there is no image data in the third frame. . Also, 22-5,23-
In No. 5, since the reference two-dimensional block is not present in the first frame, it is not blocked, and the above image data is not encoded as it is, so the image data is lost.
Therefore, in this embodiment, as shown by the arrows in FIG. 4 (b), 22-5 and 23-5 are the other ends of the same frame, that is, 22-5 is 22-1 and 23-5 is 23. It is rearranged adjacent to -1, and 21-1 is blocked using the motion vectors of 41 and 42 to form a three-dimensional block. Then, block coding is performed. At the time of decoding, the reverse procedure is used to rearrange 22-5 and 23-5 at correct positions to obtain a decoded image. By the above method, the end points of the screen will also be encoded. FIGS. 4 (c) and 4 (d) are explanatory views showing a second method for encoding the end points of the screen. Now
As shown in FIG. 4C, it is assumed that a three-dimensional block is constructed by using the method shown in the second embodiment. In the figure, the dotted lines are the boundaries of the blocks, and each block is configured along the motion vector indicated by the bold line in the figure. In FIG. 4 (c) as well, as in FIG. 4 (b), there are areas that cannot be coded at the end points 22-0, 23-0, 22-6, 23-7 of the screen. This is because the three-dimensional blocks overlap each other after the second frame. FIG. 4 (d) shows a second method for coding the end points, which is a new three-dimensional block based on the conventional method without using motion vectors as 21-0 and 21-6. This is a method of encoding separately from the three-dimensional block shown in the embodiment. At this time, 21-0 and 21-
6 is equal to 21-1 and 21-5 in the first frame, respectively.
FIGS. 4 (e) and 4 (f) are explanatory views showing another method.
In FIG. 4 (e), among the motion vectors detected as shown in FIG. 4 (c), the motion vector obtained based on the blocks 21-1 and 21-5 including the end points of the image of the first frame,
A new block 21-0, 21-6 is constructed using 24, 25, 26, 27. At this time, the reference frame is shown in FIG.
The frame is opposite in time to, that is, 23. FIG. 4 (e) is an example in which three frames form one three-dimensional block. In this case, the reference two-dimensional blocks 21-0 and 21-6 are the third frames 23-0 and 23-6. This is done because the uncoded pixels are
Because it is the most. Then motion vector 24,25,26,27
Is used to construct a three-dimensional block. Figure 4 (f)
FIG. 7 is an explanatory diagram showing details of the configuration of the three-dimensional block shown in FIG. 4 (e). In the figure, 21,22,23,21
-0 and 23-0 are the same as those in FIG. 4 (e), and the blocks corresponding to 21-0, 22-0, and 23-0 in FIG. 4 (e) are enlarged and shown. Now, with the two-dimensional block 23-0 including the end points of the third frame as a reference, a three-dimensional block is constructed using the motion vectors 24 and 25 in FIG. 4 (e). Are 44,4 respectively
There is no image data for part 3. So 44
Part of 23-0 indicated by 45, and 43 and 44 and 22-0
Are partially supplemented as indicated by arrows to form a three-dimensional block, and the three-dimensional block is block-encoded as a new block. By doing so, since the correlation in the time direction is maintained within the block, as compared with the method shown in FIG. 4 (b), the encoding of the end point block of the screen having a high compression rate and good image quality is performed. Is possible. In the third embodiment, the first method is combined with the first embodiment and the second and third methods are combined with the second embodiment. However, the present invention is not limited to this. 1 method and 2nd execution example, 2nd,
It is also possible to combine the third embodiment and the first embodiment.

Further, in the above embodiments, all the movement in the vertical direction is 0.
However, the present invention is not limited to this, and the same method can be used for the vertical direction as for the horizontal direction.

EFFECTS OF THE INVENTION As described above, according to the present invention, since a three-dimensional block is configured using motion vectors and the three-dimensional block is block-encoded, the temporal correlation can be kept high, so that the compression efficiency is effectively high. Can be realized. Moreover, since block coding is used, decoding can be performed from the middle of the image.

[Brief description of drawings]

FIG. 1 is an explanatory diagram of an image encoding method in the first embodiment of the present invention, FIG. 2 is an explanatory diagram of image data of each frame on a two-dimensional plane, and FIG. 3 is a second diagram of the present invention. FIG. 4 is an explanatory diagram of an image encoding method showing an embodiment, FIG. 4 is an explanatory diagram of an image encoding method showing a third embodiment of the present invention, and FIG. 5 is a block diagram showing a conventional interframe encoding method, FIG. 6 is a block diagram of a three-dimensional orthogonal transform coding device. 1 ... Image data of first frame, 2 ... Image data of second frame, 3 ... Image data of third frame, 4 ...
... Motion vector between the first and second frames, 5
... The motion vector between the second and third frames,
11 ... Two-dimensional block in first frame, 12 ... Two-dimensional block in second frame, 31 ... Two-dimensional block in third frame, 21 ... Part of image data in first frame, 22 ... Part of image data of 2 frames, 23 ... Part of image data of 3rd frame, 24,25,2
6,27,31〜42 …… Motion vector, 51 …… Adder, 52 ……
Quantizer, 53 ... predictor, 54 ... delay circuit, 55 ... motion detection circuit, 56 ... dequantizer, 57 ... adder, 61 ... 3
Dimensional blocking circuit, 62 ... Orthogonal transformation circuit, 63 ... Quantizer.

Claims (14)

[Claims] 1. Image data s (h,
v, t) (h: horizontal pixel index, 1 ≦ h ≦ M, v: vertical pixel index, 1 ≦ v ≦ N, t: temporal pixel index 1 ≦ t ≦ T) , A two-dimensional block sb (x, x) of m (horizontal direction) × n (vertical direction) pixels in the first frame, that is, s (h, v, 1).
y, 1) (x: horizontal block coordinates in the frame, 1 ≤
x ≦ M / m, y: vertical block coordinates in the frame, 1
≦ y ≦ N / n) and the adjacent second frames s (h, v,
2) above, the two-dimensional block sb (x, y, 2) of m × n size that is most correlated with the sb (x, y, 1) is obtained, and at the same time, the horizontal movement indicating the amount of horizontal movement Vector mvh (x, y, 2) and vertical motion vector mvv (x, y, 2) indicating the amount of motion in the vertical direction
On the next adjacent third frame s (h, v, 3),
A two-dimensional block sb (x, y, 3) having a size of mxn and a motion vector mvh (x, y, 3), mv most correlated with the sb (x, y, 2)
v (x, y, 3) is calculated, and T frame s (h, v, T)
2D block sb (x, y, t) and motion vector mvh (x, y, t), mvv (x,
y, t), and the motion vectors mvh (x, y, t) and mvv (x, y, t) for each t based on the calculated amount of motion between the two-dimensional blocks.
One representative motion vector mvhr (t), mvvr (t) is obtained for each frame, and the motion amount of the representative motion vector from the first frame is mvht (t) = mvhr (2) + mvhr (3) +. … ＋ Mvhr
(T), mvvt (t) = mvvr (2) + mvvr (3) + ... + mvvr
(T), the motion amounts mvht (t) and mvvt (t) are added to the pixel index of the image data s (h, v, t) of the t-th frame to obtain s.
(H-mvht (t), v-mvvt (t), t) and then m × n
A pixel coding method, characterized by dividing into a three-dimensional block including × T pixels, and block-coding the three-dimensional block. 2. Representative motion vectors mvhr (t), mvvr (t)
Is the motion vector mvh (x, y,
The image coding method according to claim 1, wherein histograms of t) and mvv (x, y, t) are respectively taken, and ones having a high frequency are selected from the histograms. 3. Representative motion vectors mvhr (t), mvvr (t)
Is the motion vector mvh (x, y,
The image coding method according to claim 1, wherein among t) and mvv (x, y, t), the average value is obtained by removing i from the maximum value and j from the minimum value. 4. Image data s (h,
v, t) (h: horizontal pixel index, 1 ≦ h ≦ M, v: vertical pixel index, 1 ≦ v ≦ N, t: temporal pixel index 1 ≦ t ≦ T) , M (horizontal direction) × n in the first frame, that is, s (h, v, 1)
(Vertical direction) Two-dimensional block of pixels sb (x, y, 1)
(X: horizontal block coordinates in the frame, 1≤x≤M /
m, y: vertical block coordinates in the frame, 1≤y≤N
/ n), and on the adjacent second frame s (h, v, 2), the two-dimensional block sb (x) having a size of m × n that is most correlated with the sb (x, y, 1). , y, 2), and at the same time, the horizontal motion vector mvh (s, y, 2) indicating the horizontal motion amount and the vertical motion vector mvv (x, y, 2) indicating the vertical motion amount are calculated. On the third frame s (h, v, 3) adjacent to
Two-dimensional block sb (x, y, 3) of size m × n that is most correlated with b (x, y, 2) and motion vector mvh (x, y, 3), mvv
(X, y, 3) is calculated, and motion detection is performed up to T frames s (h, v, T) in the same manner, and a two-dimensional block in each frame is obtained.
sb (x, y, t) and motion vector mvh (x, y, t), mvv (x, y,
t), and the motion vector mvh (x,
The representative motion vectors mvhr1 (t), mvhr2 (mvhr1 (t), mvhr2 (y, t), mvv (x, y, t) of which the sign is different among the center block which is the center of motion and the horizontal motion vector mvh (x, y, t) t) and the motion vector mvv (x, y, t) in the vertical direction, representative motion vectors mvvr1 (t), mvvr2 (t) with different signs are obtained for each frame, and the central block and the representative motion vector mv are obtained.
A new motion vector is generated from hr1 (t), mvhr2 (t), mvvr1 (t), mvvr2 (t) and mvh '(x, y, t), mvv'
(X, y, t), the regenerated motion vector mvh ′ (x,
y, t), mvv ′ (x, y, t), the motion amount of each block from the first frame is mvht ′ (x, y, t) = mvhr ′ (x, y, 2)
＋ mvhr ′ (x, y, 3) ＋ …… ＋ mvhr ′ (x, y, t), mvvt ′
(X, y, t) ＝ mvvr ′ (x, y, 2) ＋ mvvr ′ (x, y, 3) ＋ ……
+ Mvvr ′ (x, y, t), and the motion amount mvht ′ is added to the pixel index of the image data s (h, v, t) of the t-th frame.
(X, y, t), mvvt ′ (x, y, t) is added, and s (h + mvht ′
(X, y, t), V + mvvt ′ (x, y, t), t), and then m × n
An image coding method, characterized by dividing into three-dimensional blocks including × T pixels, and block-coding the three-dimensional blocks. 5. A motion vector mvh indicating a horizontal motion amount.
The number hz (x, t) in which (x, y, t) is 0 is calculated in the frame t for the block in the vertical direction, and the number hz (x, t) is calculated.
The horizontal block coordinate having the largest number is the horizontal block coordinate cx of the central block, and the number vz (y, t) of the motion vector mvv (x, y, t) indicating 0 in the vertical direction is 0.
t) is calculated for a horizontal block in the frame t, and the vertical block coordinates cy having the largest number vz (y, t) are set as the vertical block coordinates of the central block. The image coding method according to claim 4. 6. A motion vector mvh indicating a horizontal motion amount.
Of (x, y, t), the number hcz (x, t) in which 0 is continuous is calculated in the frame t for the block in the vertical direction, and the block coordinate in the horizontal direction in which the number hcz (x, t) is the largest is calculated. cx,
Of the motion vector mvv (x, y, t), which is the horizontal block coordinate of the central block and indicates the amount of motion in the vertical direction,
The number vcz (y, t) of consecutive 0s is calculated in the frame t for the horizontal block, and the vertical block coordinate cy having the largest number vcz (y, t) is calculated in the vertical direction of the central block. The image coding method according to claim 4, wherein block coordinates are used. 7. A representative motion vector mvhr1 in the horizontal direction.
(T) and mvhr2 (t) are the motion vectors mvh (cx-x ') of the blocks distant from the central block by the block coordinates x', x ", excluding the block including the end points of the screen among the motion vectors in each frame. , y, t), mvh (cx + x ″, y, t)
1 <y <N / n, and the histograms with the highest frequency are designated as horizontal representative motion vectors mvhr1 (t) and mvhr2 (t), and the vertical representative motion vector mvvr1 (T), mvvr2 (t)
Among the motion vectors in each frame, except for the block including the end point of the screen, the motion vector mvv of the block distant from the central block by the block coordinates y ′ and y ″.
(X, cy−y ′, t), mvh (x, cy + y ″, t) 1 <x <M / m
The histogram of the range is taken, and the one with the highest frequency in the histogram is used as the representative motion vector mvvr1 in the vertical direction.
(T) and mvvr2 (t) are set, The image coding method of Claim (4) characterized by the above-mentioned. 8. A representative motion vector mvhr1 (t) in the horizontal direction,
In mvhr2 (t), except for the block including the end point of the screen, the horizontal representative motion vector indicating the horizontal motion amount among the motion vectors in each frame is calculated as the motion of the block distant from the central block by the block coordinates x ′, x ″. Vector mvh (cx−x ′,
Of the y, t) and mvh (cx + x ″, y, t), the average values obtained by excluding i pieces from the maximum value and j pieces from the minimum value of the motion amount are representative motion vectors mvhr1 (t) and mvhr2 (h) in the horizontal direction, respectively. t), the vertical representative motion vector mvvr1 (t), mvvr
2 (t) is the motion vector mvv (x, cy-y ', t), mvh (x, cy + of the block y', y "away from the central block.
Of the y ″, t), the average value obtained by excluding i from the maximum value of the motion amount and j from the minimum value is set as the representative motion vectors mvvr1 (t) and mvvr2 (t) in the vertical direction, respectively. The image coding method according to claim 4. 9. A regenerated motion vector mvh '(x, y, t), mvv'
(X, y, t) is the block coordinate cx, cy of the central block, and the representative motion vectors mvhr1 (t), mvhr2 (t), mvvr1 (t), mv
vr2 (t), block coordinates x'when the representative motion vector mvhr1 (t) is obtained, hereinafter x "for mvhr2 (t), y'for mvvr1 (t), and for mvvr2 (t) And y ″, mvh ′ (x, y, t) = mvhr1 (t) * (cx−x) / (cx−
x ′) (where x <cx), mvh ′ (x, y, t) = mvhr2 (t) * (x−cx) / (x ″ −c
x) (where x ≧ cx), mvv ′ (x, y, t) = mvvr1 (t) * (cy−y) / (cy−
y ′) (where y <cy), mvv ′ (x, y, t) = mvvr2 (t) * (y−cy) / (y ″ −c
y) The image encoding method according to claim (4), wherein y) (where y ≧ cy) is calculated and generated. 10. A regenerated motion vector mvh '(x, y, t), mv
v ′ (x, y, t) is replaced by cx, cy, mvr1 (t), mvhr2 (t), mvvr1
Using (t), mvvr2 (t), x ′, x ″, y ′, y ″, mvh ′ (x, y, t) = mvhr1 (t) * (cx-x) ² / (cx-x ′) ² (where x <cx), mvh ′ (x, y, t) = mvhr2 (t) * (x-cx) ² / (x ″ -cx) ² (where x ≧ cx), mvv ′ (x , y, t) = mvvr1 (t) * (cy-y) ² / (cy-y ') ² (where y <cy), mvv' (x, y, t) = mvvr2 (t) * (y- The image coding method according to claim (4), wherein the image coding method is performed by calculating as follows: cy) ² / (y ″ -cy) ² (where y ≧ cy). 11. A motion vector, and a frame serving as a reference of the motion vector is image data s (h, v, T) of the Tth frame.
, And image data s (h,
v, T-1), the motion is detected, and the highest correlation m × n
The image coding according to claim (1) or (4), wherein a two-dimensional block having a size of 1 is obtained, and then motion detection is similarly performed up to the first frame s (h, v, 1). Method. 12. A block including a horizontal or vertical end point of a three-dimensional block is configured as a three-dimensional block with a motion vector set to 0 between consecutive T frames, and the three-dimensional block is block-encoded. The image coding method according to claim 1, which is characterized in that: 13. If three-dimensional blocks cannot be formed into blocks because there is no pixel data, the three-dimensional blocks are formed by retaining the pixels in the same frame that lack pixels, and the three-dimensional blocks are block-encoded. The image coding method according to claim 1, wherein the image coding method comprises: 14. If three-dimensional blocks cannot be formed into blocks because there is no pixel data, the three-dimensional blocks are constructed by holding the pixels of other frames in other frames,
The image coding method according to claim 1, wherein the three-dimensional block is block-coded.