JPH0366278A - Highly efficient coding method for video signal - Google Patents

Abstract

PURPOSE:To easily apply special reproduction such as search by taking a difference between a picture interpolating a decoded 1/4 picture into twice longitudinally and laterally respectively and an original picture, coding and decoding the interpolation difference signal and sending and recording the result of coding of the interpolation difference signal and the 1/4 picture signal. CONSTITUTION:A decoded 1/4 picture signal stored in a 1/4 frame memory 8 is interpolated to the size by original one frame with an interpolation filter 5. When a subtractor 10 takes a difference from a signal in a frame memory 1, an interpolation difference signal of each block is accumulated by an accumulation adder 23. The result of accumulation is compared with a predetermined threshold level at a discriminator 24 to discriminate whether or not an interpolation difference signal of each block is to be coded. When a stop code is sent to a decoder, a decoded signal is the interpolated decoding 1/4 picture signal only. Thus, a bit number is reduced by generating a block without sending an interpolation difference signal.

Description

[Detailed description of the invention] Industrial applications The present invention relates to a high efficiency encoding method for video signals.

With video signals such as conventional technology television signals (above two consecutive
When the time difference between frames is small, the change in the image between these frames is considered to be small, but interframe coding (
The purpose of this is to utilize such inter-frame correlation to improve efficiency, and usually the difference signal between two consecutive frames is encoded and transmitted, and the receiving side uses the inter-frame difference value. It reproduces the television signal by accumulating the .

In addition to this method, motion compensation, interframe/intraframe adaptive coding, etc. have been devised to prevent sudden increases in difference values due to motion.

Problems to be Solved by the Invention However, it is generally difficult to easily perform special playback using conventional interframe coding, or interframe/intraframe adaptive coding. Ah1゜For example, if you want to search for an arbitrary frame, you must first go back and decode the original signal part from which the inter-frame difference is taken, and then restore the screen from there depending on the search speed. Signal part and G; Lpcm
Refers to the part that is refreshed using intra-frame encoding, which is relatively less affected by deterioration and error propagation.

Furthermore, although hierarchical encoding methods are known for still image wiring transmission, a method called the Laplacian pyramid encoding method shown in FIG. 7 will be described as an example.

First, the original image Qe is subjected to range restriction and sub-sampled, and the resultant image is defined as a Gaussian plane G1. This process is repeatedly applied to obtain a Gaussian plane G1)=G-1. Next, each Gaussian plane G1 is interpolated, and the result obtained by subtracting this from the Gaussian plane G1-1 at the lower level is defined as the Laplacian plane L1-1,
Obtained Laplacian pyramid La~I, m-+
are encoded and transmitted sequentially from the upper layer.

This method can be displayed in stages according to various screen sizes, and is considered effective for high-speed searches even when applied to moving images.If the conventional method is applied to moving images as is, The data rate may become too high.

In view of these points, it is an object of the present invention to provide a high-efficiency encoding method suitable for special playback such as search, and a hierarchical encoding method suitable for application to moving images.

Means for Solving the Problems According to the present invention, for the first frame to be encoded or the frame to be refreshed, 1/4 screen obtained by subsampling at 2:1 in the vertical and horizontal directions is encoded and decoded. The result is called the decrypted 1/4 screen.
The difference between the screen obtained by interpolating this decoded 1/4 screen twice in the vertical and horizontal directions and the original screen is taken, and this difference signal is called an interpolated difference signal, and this interpolated difference signal is encoded and decoded. For subsequent frames, the 1/4 screen obtained by subsampling 2:1 vertically and horizontally, and the decoding of the previous frame will be transmitted and recorded. Find the difference signal between the converted 1/4 screen, encode and decode it, and add the decoded result of this difference signal and the decoded 1/4 screen of the previous frame to find the decoded 174 screen of the frame. The difference between the screen obtained by interpolating the decoded 1/4 screen twice vertically and horizontally and the original screen is obtained, and the second-order difference signal between this interpolated difference signal and the decoded interpolated difference signal of the previous frame is obtained. This is a highly efficient video signal encoding method characterized by encoding and decoding this, and transmitting and recording the encoding result of this secondary difference signal and the encoding result of the above-mentioned 1/4 screen difference signal.

In addition, repeat subsamples multiple times vertically and horizontally to
/4-frame sleep 1/16 screen, 1/64-frame direct/both sides/4N screen are obtained, and operations similar to claim (1) are performed in order from the smallest screen, that is, 1/4N screen and decoding of the previous frame. Find the difference signal with the 1/4N screen and encode it,
By adding the decoding result of this difference signal and the decoded 1/4" screen of the previous frame, a decoded 1/4N screen of the frame is obtained, and this decoded 1/4" screen is divided vertically and horizontally into two The interpolated screen and 1/4 (NN-1
) screen difference, obtain a second-order difference signal between this interpolated difference signal and the decoded interpolated difference signal of the previous frame 1/4 (NN-1) screen, and decode it (L). This is a high-efficiency encoding method for video signals that is characterized by repeating the following steps. When encoding a quarter screen, the screen is divided into multiple blocks, the average value within each block is determined, and the signal within each block is This is a highly efficient encoding method for video signals, which is characterized by subtracting the average value of the same block from , then performing inter-frame difference and encoding (＼The obtained average value is transmitted and recorded separately. When calculating the difference between frames, the screen is divided into multiple blocks.
Calculate the motion vector for each block and perform motion compensation. (4) When calculating the inter-frame difference of the 1/4 screen, the motion vector of the block of the 1/4 screen at the corresponding position is used to calculate the motion. This is a highly efficient encoding method for video signals characterized by performing compensation. Also 1/4(N'
Divide the screen into multiple blocks when encoding the interframe difference signal or interpolation difference signal of the screen 1. , 1/4N pixels For each of the interpolated difference signals, a certain threshold is set for the sum or average value of the interframe difference signals in each block, and a certain threshold value is set for the sum or average value of the intraframe signals in each block. This is a high-efficiency video signal encoding method characterized by determining and switching between the encoding result of the inter-frame difference signal and the encoding result of the intra-frame signal to be transmitted by determining and comparing these results. . In addition, when switching between transmitting the interframe difference signal as fresh 9- 0- 1- direct 1/4 screen encoding result and interpolated difference signal, 1/4 screen intra-frame encoding result This is a high-efficiency video signal encoding method characterized by shifting a frame for transmitting the intraframe encoding result of the interpolated difference signal and a frame for transmitting the intraframe encoding result of the interpolated difference signal by a certain number of frames. Also
When encoding an interpolated difference signal, the screen is divided into multiple blocks, a certain threshold is set for the sum, average value, or cumulative sum of the squares of the interpolated difference signals within the block, and if it is less than that threshold, the interpolated difference is This is a highly efficient video signal encoding method characterized by switching so that the signal is not transmitted. Also, when encoding interframe difference signals or interpolation difference signals, the screen is divided into multiple blocks and a certain threshold is set for the sum, average value, or cumulative sum of squares of the secondary difference signals within the block. ,
This is a high-efficiency video signal encoding method characterized by switching not to transmit the secondary difference signal when it is less than the threshold value.

Effect of the Invention The present invention enables hierarchical encoding while suppressing the amount of information by using the above-mentioned means, and can handle special playback such as searches more easily than conventional methods by decoding up to the hierarchy according to the speed. be able to. You can also switch layers depending on the size of the playback screen and the required image quality. Furthermore, by shifting the frames to be refreshed for each layer, it is possible to concentrate the number of bits in a certain frame, and it is possible to suppress variations in image quality. Furthermore, by generating blocks that do not require transmission of interpolated difference and secondary difference signals, the number of rebits can be reduced.

Furthermore, there is no need to transmit motion vectors for each screen size, and the number of bits can be reduced.

EXAMPLES Below, examples of the present invention will be described with reference to the drawings.

FIG. 1 is a diagram for explaining the encoding method in the first embodiment of the present invention. In the figure, 1, 1, 12-13 are frame memories, 2 is a band-limiting filter, 3 is a sub-frame memory, 4.8 is a 1/4 frame memory, 5 is an interpolation filter, 6, 1) is an encoder 7, and 12 is a decoder. transformation machine
9, 16, and 17 are switches, lOl 14, 15 are subtractors @18, 19 are adders 20, and 21 is a buffer memory.

The image signal for l frames stored in the frame memory 1 is band-limited by a band-limiting filter 2, and then subsampled to 1/2 vertically and horizontally by a sub-sampler 3, and stored in a 1/4-frame memory 4.

In the first frame to be encoded or in the frame to be refreshed (switches 9, 16, 1
7 are connected to the a side, and this 1/4 screen signal (1/4 screen signal) is encoded by the encoder 6 and outputted, decoded by the 1/4 frame decoder 7, and stored in the 1/4 frame memory 8. .

Next, the switch 9 is connected to the b-IL switch 16 is connected to the C side, and the decoding 1 stored in the 1/4 frame memory 8
/4 screen signal (Y) is interpolated to the size of the original l frame by the interpolation filter 5, and the difference with the signal in the frame memory l is taken by the subtracter lO.

This interpolated difference signal is encoded and output by the encoder 1), decoded by the decoder 12, and stored in the frame memory 13.

From the first frame to be encoded or in a frame other than the frame to be refreshed.
Switches 16 and 17 are each connected to the b side, and the signal of the nth frame stored in frame memory 1 is subsampled and stored in 1/4 frame memory 4. (n-1)
A subtracter 14 calculates the difference between the frame and the decoded 1/4 screen signal. This 1/4 screen inter-frame difference signal ζ is encoded by the encoder 6 and output.
It is decoded by the decoder 7 and added by the adder 18.
−1) Added to the decoded 1/4 screen signal of frame nth
The decoded frame is stored in the 1/4 frame memory 8 as a 174 screen signal.

Next, the switch 9 is connected to the B side, and the switch 16 is connected to the C side. The subtracter 10 calculates the difference between the signal and the signal in the frame memory 1.

The second-order difference between this interpolated difference signal and the decoded interpolated difference signal of the (n-1)th frame stored in the frame memory 13 is obtained by the subtracter 15, encoded by the encoder 1), and output. It is decoded by the Shizu decoder 12, added to the decoded interpolated difference signal of the (n-1)th frame by the adder 19, and stored in the frame memory 13 as the decoded interpolated difference signal of the n-th frame.

Note that in this embodiment, an encoder and decoder for the 1/4 screen and interpolated difference signals are installed separately, and switches and buffer memories are installed before and after the encoder.
1 by controlling the timing of input and output to the decoder.
A pair of encoders and decoders can be shared.

Also, by combining similar configurations, 1/16
It is also possible to separate and send higher-level images such as ffi 1/84 screen.

FIG. 2 is a diagram for explaining the encoding method in the second embodiment of the present invention. In the figure, 22 is a blocking slider, 23 is an accumulative adder, 24 is a determination method, and 25 is a truncated code generator.

The 1/4 screen signal in the n-th frame stored in the 1/4 frame memory 4 is divided into small blocks by the block generator 22. Here, the size of the block is 8*8. Decoded 1/4 stored in 1/4 frame memory 8
When the screen signal is interpolated to the original size of one frame by the interpolation filter 5 and the difference from the signal in the frame memory l is taken by the subtracter 10, the interpolated difference signal of each block is added by the cumulative adder 23. By comparing the cumulative result with a predetermined threshold value by the determiner 24, it is determined whether or not to encode the interpolated difference signal of each block.If not, the truncation code generator 25 Send a truncation code.

On the decoder side (if a truncated code is sent), the interpolated difference signal is not used for that block, and only the interpolated version of the decoded 1/4 screen signal is used as a composite signal. This generates blocks that do not require transmission of interpolated differential signals, thereby reducing the number of bits.

In this embodiment, the force t is determined based on the cumulative result of the interpolated difference signal using a cumulative adder that does not depend on the block size. The average value or the accumulation of squares may be used. FIG. 3 is a diagram for explaining the encoding method in the third embodiment of the present invention. In the figure, 26 is a blocked song, 27 is a cumulative adder, and 28 is a judger.

When the subtracter 15 calculates the second-order difference between the interpolated difference signal obtained by the subtracter IO and the decoded interpolated difference signal of the (n-1)th frame stored in the frame memory 13,
A cumulative adder 27 accumulates the secondary difference signals of each block, and a determiner 28 compares the cumulative result with a predetermined threshold to determine whether or not to encode the interpolated difference signal of each block. If not encoded, the truncation code generator 25 sends a truncation code instead.

On the decoder side (1) if a truncated code is sent.
The second-order difference signal is not used for the other block, and the interpolation difference signal of the previous frame is used as the interpolation difference signal of that frame. This generates blocks that do not require transmitting secondary differential signals, thereby reducing the number of bits.

The strength is determined from the cumulative result of the secondary difference signal using an accumulator. FIG. In the figure, 29 is an average value separation method, and 30 is a buffer memory.

The 1/4 screen signal in the n-th frame stored in the 1/4 frame memory 4 is divided into small blocks by the block generator 24. Here, the size of blocks 17 to 8- is 8*8. Next, the average value in each block is calculated by the average value separator 29. After subtracting the average value of that block from the signal in each block, the average value is stored in the buffer memory 30. The signal after the average value separation is sent to the encoder. 6 respectively.

After the signal sent to the encoder 6 is decoded, the average value separated from each block is added to restore it as a decoded signal.This allows each block to be encoded by transform coding etc. In some cases, such as when all the values in a block are close to the average value, the number of bits is reduced by creating blocks that do not require transmitting the signal after the average value separation, as in the second and third embodiments. can be achieved.Also
This average value can be regarded as an upper layer for 1/4 screen (1/64 screen for 8*8 pixel blocks). This is a diagram. As shown in the figure, when switching to directly transmit the 1/4 screen encoding result and interpolated difference signal as periodic refresh, the frame and interpolated difference signal that transmit the 1/4 screen intraframe encoding result are transmitted. The frame for sending the intraframe encoding result is shifted by a certain number of frames. This results in
It is possible to distribute the necessary number of bits in the refresh frame portion, and it is possible to prevent variations in image quality between the refresh frame and other frames.

FIG. 6 is a diagram for explaining the encoding method in the sixth embodiment of the present invention. In the figure, 31 is a motion vector detection method 32, 33 is a prediction block extraction method, and 34 is a motion vector expansion method.

When the subtracter 14 calculates the inter-frame difference of the 1/4 screen, a motion vector is obtained for each block by 31, and using this, the signal of the block at the position moved by the motion vector is extracted by 32, and this signal and Send the difference between the two to the encoder.

Furthermore, (4) L when calculating the difference between frames of 1/4 screen.
Motion compensation is performed by enlarging the motion vector by 34 using the motion vector in the 1/4 screen block at the corresponding position.

Thereby, there is no need to transmit motion vectors for each size of screen, and the number of bits can be reduced.

As described in detail, according to the present invention, hierarchical encoding is possible while suppressing the amount of information, and even during special playback such as searching, by decoding up to the layer according to the speed, it is more efficient than conventional methods. can be easily accommodated. In addition, the layers can be switched depending on the size of the screen to be played and the required image quality. By shifting the refresh frame for each layer, it is possible to concentrate the number of bits in a certain frame, reducing variations in image quality. Even if it can be suppressed

[Brief explanation of drawings]

FIG. 1 is a block diagram showing the encoding method of the first embodiment of the present invention. FIG. 2 is a block diagram of the second embodiment of the present invention. FIG. 3 is a block diagram of the third embodiment of the present invention. Fig. 4 is a block showing a fourth embodiment of the present invention @ Fig. 5 is a block showing a fifth embodiment of the present invention @ Fig. 6 is a block showing a fifth embodiment of the present invention FIG. 7 is a diagram for explaining the conventional Laplacian pyramid encoding method. 1.13...Frame memory, 2...Band limit filter 3...Sub sampler 4, 8...1/4
Freno\Memory, 5... Complementary Philkyu 6, 1)
... Encoder 7, 12 ... Decoding method 9, 16,
17... Switch, 22, 28... Block generator 23, 27... Accumulator adder 29... After average value separation 31... Motion vector detection edge 32, 33... Block extractor 34 ...Motion vector magnifier.

Claims (8)

[Claims] (1) For the first frame to be encoded or the frame to be refreshed, two
: Encode the 1/4 screen obtained by subsampling to 1,
The result is called a decoded 1/4 screen, and the difference between the screen obtained by interpolating this decoded 1/4 screen twice vertically and horizontally and the original screen is taken, and this difference signal is called an interpolated difference signal. , this interpolation difference signal is encoded and decoded, and the encoding result of the interpolation difference signal and the encoding result of the 1/4 screen are transmitted and recorded.For subsequent frames, 2: Find the difference signal between the 1/4 screen obtained by subsampling to 1 and the decoded 1/4 screen of the previous frame, encode and decode it, and combine the decoding result of this difference signal with the decoded 1/4 screen of the previous frame. The decoded 1/4 screen of the frame is obtained by adding the decoded 1/4 screen, and the difference between the original screen and the screen obtained by interpolating this decoded 1/4 screen twice vertically and horizontally is calculated. A secondary difference signal between the difference signal and the decoded interpolated difference signal of the previous frame is obtained, encoded and decoded, and the encoding result of this secondary difference signal and the above-mentioned 1/4 screen difference signal are obtained. A highly efficient encoding method for video signals, characterized by transmitting and recording encoding results. (2) Repeat the sub-samples multiple times vertically and horizontally to
4 screen, 1/16 screen, 1/64 screen,...1/4^N
Find the screen and perform the same operation as in claim (1) starting from the smallest screen, that is, find the difference signal between the 1/4^N screen and the decoded 1/4^N screen of the previous frame, and encode this. By adding the decoding result of this difference signal and the decoded 1/4^N picture of the previous frame, the decoded 1/4^N picture of the frame is obtained, and this decoded 1/4^N
A screen that interpolates the screen twice vertically and horizontally, and a 1/4
(＾N＾−＾1＾) screen difference is taken, and the second-order difference between this interpolation difference signal and the decoded interpolation difference signal of the previous frame 1/4＾(＾N＾−＾1＾) screen is calculated. 2. The highly efficient encoding method of a video signal according to claim 1, wherein the operation of obtaining a signal, encoding and decoding the same is repeated. (3) When encoding a 1/4 screen, divide the screen into multiple blocks, find the average value within each block, subtract the average value of the same block from the signal within each block, and then calculate the interframe difference. 2. The high-efficiency encoding method of a video signal according to claim 1, wherein the average value obtained is separately transmitted and recorded. (4) When calculating the interframe difference of a 1/4 screen, divide the screen into multiple blocks, calculate the motion vector for each block, perform motion compensation, and then calculate the interframe difference of the interpolated difference signal. A highly efficient encoding method for a video signal, characterized in that motion compensation is performed using a motion vector in a 1/4 screen block at a corresponding position. (5) When encoding the interframe difference signal of the 1/4^N screen or the interframe secondary difference signal of the interpolated difference signal, the screen is divided into multiple blocks, and the 1/4^N screen, the interpolated difference signal For each signal, a certain threshold is set for the sum or average value of the interframe difference signals in each block,
Also, by calculating the sum or average value of the intra-frame signals in each block and comparing them, it is determined and switched which of the inter-frame difference signal encoding result and the intra-frame signal encoding result is to be transmitted. 3. The high-efficiency encoding method of a video signal according to claim 1 or 2, characterized in that: (6) A frame that sends the intra-frame encoding result of 1/4 screen when switching to directly transmit the 1/4 screen encoding result and interpolated difference signal without sending the interframe difference signal as periodic refresh. 3. The high-efficiency encoding method of a video signal according to claim 1, wherein frames for transmitting intraframe encoding results of the interpolated difference signal and the interpolated difference signal are shifted by a certain number of frames. (7) When encoding an interpolated difference signal, if the screen is divided into multiple blocks and a certain threshold is set for the sum, average value, or cumulative sum of squares of the interpolated difference signals within the block, and the value is less than or equal to that threshold. 3. The high-efficiency encoding method of a video signal according to claim 1, further comprising switching the interpolation difference signal not to be transmitted. (8) When encoding the interframe secondary difference signal of the interpolated difference signal, the screen is divided into multiple blocks, and a certain threshold is set for the sum, average value, or cumulative sum of the squares of the secondary difference signals within the block. 3. The high-efficiency encoding method of a video signal according to claim 1, further comprising the step of: providing a threshold value, and switching not to transmit the second-order difference signal when the difference is less than the threshold value.