JP3181002B2 - Free frame drop video coding method - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method for realizing frame dropping of coded data which has been coded once, decoding the coded data, returning the decoded data to a moving image, and performing frame dropping again. The present invention relates to free-moving-frame moving image encoding that simply realizes a missing frame by simply removing a part of the encoded data itself instead of the encoding procedure (see FIG. 1).

[0002]

2. Description of the Related Art Conventionally, when an error occurs during transmission of coded data by using a free frame dropping encoding method, data containing an error is appropriately discarded so that a moving image can be obtained without causing a broken image. The image can be reproduced.

Further, when the decoding processing amount such as decoding by software is insufficient, the encoded data is appropriately discarded to enable real-time reproduction. Furthermore, when networks having different transmission speeds are cascaded, or when the same decoded data is simultaneously (broadcast) input to decoders having different decoding processing capabilities, the decoded data is not transmitted during the transmission or during the decoding process. If it has to be discarded, the decoding process often has to be omitted (see FIG. 4).

Conventionally, in order to realize the above-mentioned free dropping of the pieces,
All frames are encoded in intra-frame mode.

In order to reduce the amount of information generated by moving picture coding, in an area where images are the same in consecutive frames, an inter-frame copy mode in which the contents of the previous frame are used as they are in the current frame, Inter-frame coding techniques such as an inter-frame difference mode for coding only the difference between the current frame and the current frame, and motion compensation for coding motion information for each block are used.

[0006]

However, in order to realize the above-mentioned conventional free frame dropping, in the technique of encoding all frames in the intra-frame mode, even if the same image is used in successive frames, the encoding is performed one by one. Since data is generated, there is a problem that the information amount of the encoded data increases.

Further, in order to reduce the amount of information generated in the moving picture encoding, in an area where the picture is the same in consecutive frames, an inter-frame copy mode in which the contents of the previous frame are used as they are in the current frame. In the inter-frame coding mode such as the inter-frame difference mode for coding only the difference between the current frame and the current frame, and the motion compensation for coding motion information for each block, it is assumed that all frames are decoded. If the coded data is missing due to circumstances, the output image thereafter will be broken.

As a countermeasure against errors when the inter-frame coding technique is used, there is a technique called refresh (constant cycle) in which all blocks are intra-frame coded (rewrite mode) at fixed intervals. Even if the data is lost, normal decoding cannot be performed until the next refresh occurs after the data is lost. On the other hand, if the refresh cycle is shortened, intra-frame coding is frequently used, and the amount of generated information increases.

[0009] As another method for realizing free frame drop, a key frame to be decoded is always set at a certain period, and a frame between key frames is copied between frames from the immediately preceding key frame. In this case, it is conceivable to perform encoding by performing inter-frame difference, motion compensation, and the like. In this case, the decoding side requires two image memories, a key frame image memory and a current frame image memory. There was a problem that it became large.

[0010] Further, when performing broadcast, the number of decoders is larger than the number of encoders, so that the economical efficiency of the decoder is a major problem. For this reason, it is conceivable that the decoding process is performed by a main central processing unit (CPU) of a general-purpose personal computer. Therefore,
The decoding process must be as simple as possible.

SUMMARY OF THE INVENTION The present invention has been made to solve the above problem, and an object of the present invention is to provide a technique capable of performing normal decoding even when encoded data is lost. Is to do.

Another object of the present invention is to provide a technique capable of reducing the amount of generated information.

Another object of the present invention is to provide a technique capable of easily performing a decoding process.

The above and other objects and novel features of the present invention will become apparent from the description of the present specification and the accompanying drawings.

[0015]

The outline of a typical invention among the inventions disclosed in the present application will be briefly described as follows.

The input image is divided into rectangular blocks, and for each block, a rewrite mode for encoding the image of the block or a copy mode in which the block at the same position in the previous frame is determined as a block is determined. A code indicating whether the mode of the block is the rewrite mode or the copy mode is output, encoded data is output for the block in the rewrite mode, and all blocks are always encoded in the rewrite mode at a constant period (refresh). A mode flag indicating whether a rewrite mode or a copy mode is stored for each frame in a flag memory for each frame, and coded data generated in the rewrite mode is stored in a block coded data memory. One frame is accumulated for each block, and at the end of a certain period (refresh) When the lag memory is cleared in the copy mode and thereafter enters the rewrite mode until the next fixed cycle (refresh), it is retained, and in the rewrite mode, the coded data of the block is stored in the block coded data memory. Overwriting, reading the contents of the flag memory in order for each block, outputting a code indicating the same in the copy mode, outputting the code indicating the same in the rewrite mode and the contents of the block coded data memory of the corresponding block, At the end of a fixed period (refresh), the flag memory is cleared in the copy mode. Thereafter, at the next fixed period (refresh), a code indicating the output is output to generate an encoded data stream. At the time of transmission and decoding, the fixed period is set. (Refresh) coded data is always transmitted and decoded, and a fixed period (refresh ) Other than the coded data, transmission, or is missing appropriately data by the state of the decoding process, characterized by omitting the decoding process.

[0017]

According to the above-mentioned means, coded data including all portions changed from the time of refresh to the current frame is output to each frame by the effect of the flag memory and the block coded data memory. Even if the other data is dropped, the data of the subsequent frame is not disturbed.

Although the generated code amount is substantially proportional to the number of blocks output in the rewrite mode, the method of the present invention
Once in the rewrite mode, the mode is maintained until the next refresh, so that the number of rewrite mode blocks increases for each frame during the refresh. However, usually, in a moving image, a moving area in a screen is limited, and therefore, there are few blocks that enter a rewrite mode or a copy mode between refreshes. Therefore, even if the method of the present invention is used, the increase in the code amount is small.

According to the method of the present invention, since only one frame memory is required for the decoder, the structure of the decoder is simplified. In the copy mode, there is no need to access the frame memory, so that the processing amount is greatly reduced.

[0020]

Embodiments of the present invention will be described below in detail with reference to the drawings.

FIG. 1 is an explanatory diagram for explaining an embodiment of a free frame dropping moving picture encoding method according to the present invention.
(A) is a flag memory for one frame, and (b) is a block coded data memory for one frame. In the figure, "0" is a flag for the copy mode, and "1" is a flag for the rewrite mode. (B) In the figure, each block contains encoded data for one block.

The processing procedure of the free frame dropping moving picture encoding method according to the present embodiment will be described.

(1) The encoder divides an input image into blocks, and determines, for each block, whether the block is an inter-frame copy mode (copy mode) or a rewrite mode. For this determination, for example, the sum of the squared differences between the decoded previous frame and the input frame is calculated, and the block whose value is equal to or less than a certain threshold value is set to the copy mode, and if not, the rewrite mode is set. Rewrite mode blocks are encoded by existing methods such as discrete cosine transform (DCT) encoding and vector quantization.

(2) All blocks are always coded in the rewrite mode at regular intervals (refresh).

(3) A flag memory for storing a mode flag indicating a rewrite mode or a copy mode for one frame for each block, and a block coding for storing block coded data generated in the rewrite mode for one frame for each block. Prepare a data memory (see FIG. 1). At the end of the refresh, the flag memory is cleared in the copy mode, and thereafter, if the rewrite mode is once set until the next refresh, the flag memory is retained (even if the copy mode is set in the frame after the rewrite mode, the rewrite mode is not changed). Leave). In rewrite mode,
The coded data of the block is overwritten on the block coded data memory.

(4) The contents of the flag memory are sequentially read out for each block, and the code indicating the code is transmitted in the copy mode, and the content of the block coded data memory of the block corresponding to the code is transmitted in the rewrite mode. I do.

(5) At the time of refresh, a code indicating the refresh is transmitted.

(6) The decoder prepares an image memory for one frame, and sequentially decodes the input coded data. Specifically, when the code indicating the copy mode is input, the process proceeds to the next block without changing the image memory, and when the code indicating the rewrite mode is input, the corresponding block encoded data is decoded and overwritten on the image memory. .

(7) When frames of encoded data are dropped, data other than the encoded data for refresh is symmetrical (data for refresh is not dropped).

FIG. 2 shows an apparatus (apparatus comprising a computer) for carrying out the free-frame-falling moving picture encoding method of the present embodiment.
FIG. 2 is a block diagram showing a functional configuration of one embodiment;
1 is a block division unit, 12 is a frame counter, 13 is a mode determination unit, 14 is an image memory, 15 is a block decoding unit, 16 is a block encoding unit, 17 is a flag memory,
8 is a block coded data memory, 19 is a multiplexing unit, 2
1 is a separation unit, 22 is a block decoding unit, 23 is an image memory, and 30 is a frame drop unit.

The operation of the present embodiment will be described with reference to FIG.

The input image is input to the block division unit 11, and the subsequent processing is performed for each block. Each block is determined by the mode determination unit 13 to be in the copy mode or the rewrite mode. The frame counter 12 counts the number of input frames and outputs a refresh signal at a certain cycle. At the time of refresh, all blocks are determined to be in the rewrite mode. In cases other than refresh, the mode determination unit 13 compares the input block with the corresponding block in the image memory and outputs a mode determination result.

As a mode determining method, for example, when the sum of squares of each pixel between the input block and the block before one frame is larger than a certain value, the rewriting mode is set, and when the sum is smaller, the copy mode is set.

When the mode is rewritten, the block encoder 16 converts the input block into a discrete cosine transform (DCT).
Encoding is performed by a system or a method such as vector quantization, and block code data is output. When the mode is rewriting, the block decoding unit 15 reproduces the block from the block coded data and rewrites the corresponding block in the image memory.
The flag memory 17 is cleared in the copy mode when the refresh is completed. After the refresh, when a certain block is in the rewrite mode, its flag is stored in memory. In the copy mode, the flag is stored only when the contents of the memory are in the copy mode. The block coded data is written to the block coded data memory each time it is generated. The multiplexing unit 19 multiplexes the refresh signal with all the contents of the flag memory and, when the flag memory indicates the rewrite mode, the contents of the corresponding block coded data memory.

In the frame drop unit 30, as shown in FIG.
If dropping is required for some reason, the data in the encoded data stream is dropped as needed. However, refresh data is not lost.

On the other hand, the code data is separated by the separation unit 21 into a mode flag, block coded data, and a refresh signal. The refresh signal is not used in the decoding unit. The block decoding unit 22 looks at the mode flag and moves to the next block in the copy mode, and decodes the block coded data to the image memory 23 in the rewrite mode. The decoded image is output from the image memory 23.

In the above-described embodiment, the refresh is performed in units of frames. However, it is apparent that the present invention can appropriately divide an image and perform the image in units of the divided images.

Further, in the above-described embodiment, when the encoded data is lost, the encoded data is lost in units of frames. However, the image is appropriately divided, and header information is added for each division unit, and the division is performed on the decoding side. Obviously, if the position of the unit on the image can be determined, data can be deleted in the divided unit.

As described above, the present invention has been specifically described based on the above-described embodiment. However, the present invention is not limited to the above-described embodiment, and it is needless to say that various changes can be made without departing from the scope of the present invention. is there.

[0040]

The effects obtained by typical aspects of the invention disclosed in the present application will be briefly described as follows.

(1) Due to the effects of the flag memory and the block coded data memory, coded data including all portions changed from the time of refreshing to the current frame is output to each frame. Even if a frame is dropped, the data of the subsequent frame is not disturbed.

(2) Since the number of blocks that enter the rewrite mode or the copy mode between refreshes is small, the increase in the code amount is small.

(3) Since only one frame memory is required for the decoder, the configuration of the decoder is simplified.

(4) In the copy mode, since it is not necessary to access the frame memory, the processing amount is greatly reduced.

(5) Even if a frame is dropped, a stable output image (an image following the frame drop can be normally reproduced) can be output, so that the decoding process can be lightened.

[Brief description of the drawings]

FIG. 1 is an explanatory diagram for explaining an embodiment of a free frame dropping moving picture encoding method according to the present invention.

FIG. 2 is a block diagram showing a functional configuration of one embodiment of an apparatus (apparatus including a computer) for executing the free frame dropping moving picture encoding method of the present embodiment.

FIG. 3 is a diagram for explaining the concept of free drop encoding.

FIG. 4 is a diagram for explaining a case where free dropout encoding is required.

[Explanation of symbols]

11: block dividing unit, 12: frame counter, 13
... Mode deciding unit, 14 ... Image memory, 15 ... Block decoding unit, 16 ... Block encoding unit, 17 ... Flag memory,
18: block coded data memory, 19: multiplexing unit,
21: separation unit, 22: block decoding unit, 23: image memory, 30: frame drop unit.

Claims (1)

(57) [Claims] 1. An input image is divided into rectangular blocks, and for each block, a rewrite mode in which an image of the block is encoded or a copy mode in which a block at the same position in a previous frame is used as the block is determined. Then, a code indicating whether the mode of the block is the rewrite mode or the copy mode is output, coded data is output for the block in the rewrite mode, and all blocks are always set in the rewrite mode at regular intervals (refresh). In a moving picture coding method for coding, a mode flag indicating a rewrite mode or a copy mode is accumulated for one frame in each block in a flag memory, and coded data generated in the rewrite mode is stored in a block coded data. One frame for each block is stored in the memory, and at the end of a certain period (refresh) When the flag memory is cleared in the copy mode and thereafter enters the rewrite mode until the next fixed cycle (refresh), it is retained, and in the rewrite mode, the coded data of the block is stored in the block coded data memory. The contents of the flag memory are sequentially read out for each block, a code indicating the contents is output in the copy mode, and the code indicating the contents is output in the rewrite mode, and the contents of the block coded data memory of the corresponding block are output. The flag memory is cleared in the copy mode at the end of a fixed period (refresh), and thereafter, at the next fixed period (refresh), a code indicating the output is output to create an encoded data stream. The encoded data for the cycle (refresh) is always transmitted and decoded, and the ) Except the coded data, transmission, or appropriate to the missing data by the state of the decoding process, universal frame dropping video encoding method characterized by omitting the decoding process.