JP3017646B2 - Image decoding device with frame rate conversion function - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image decoding apparatus with a frame rate conversion function capable of decoding an image at a frame rate different from that of an encoded image signal.

[0002]

2. Description of the Related Art When digitally represented image data is transmitted or stored, encoding is performed to reduce the amount of data. As an encoding method, there is a method of reducing redundancy by utilizing temporal or spatial correlation of image information (image data).

As a method utilizing temporal correlation, there are a method of encoding a difference between two consecutive screens (frames) and a method of detecting a motion of an image to perform motion compensation. As a method using spatial correlation, an image is divided into blocks of a predetermined size (for example, 8 pixels each in both the vertical and horizontal directions), the data in the blocks are orthogonally transformed, and the transform coefficients are scan-converted. Some (for example, rearrange from low frequency components to high frequency components) and perform variable length coding. An image coding system (hereinafter, abbreviated as MPEG2), which is being standardized by the Moving Picture Experts Group (MPEG), uses the above two methods in combination.
This is the provisional recommendation of MPEG2 "Generic Coding of Movi
ng Pictures and Associated Audio ”
It is described in IEC1381 8 -2.

FIG. 3 shows an example of the configuration of a conventional image decoding apparatus, which comprises decoding processing means A and display processing means B. An input bit stream 100, which is a bit string of encoded data, is input to the decoding processing unit A, and after decoding, a reproduced image 200 is output. The display processing means B converts the reproduced image 200 so as to be suitable for a display monitor, and outputs an image output 300. In MPEG2, the contents of processing to be performed by the decoding processing means A are specified, and the display processing is not specified.

FIG. 4 shows an example of the configuration of the image decoding processing means A of FIG. 4, a buffer control unit 10, a variable length decoder 20, a scan converter 30, an inverse quantizer 40,
A decoding process is executed by the inverse DCT unit 50 and the motion compensation image reproducing unit 60. 71 is a buffer memory;
73 and 74 are frame memories (three I, P,
B frame memory). Numerals 81 and 82 denote data, 83 and 84 denote predicted frame data or predicted field data, 85 denotes reproduced pixel data, 86 denotes B frame image data, 100 denotes an input bit stream representing an encoded image, 200 Indicates a reproduced image.

Next, the operation will be described. The input bit stream 100 is stored in the buffer memory 71 as data 81 under the control of the buffer control unit 10. Data 82 read from buffer memory 71
Is variable-length decoded by the variable-length decoder 20.

Although not all data is variable-length coded, fixed-length codes are also decoded by the variable-length decoder 20. Next, after the scan converter 30 rearranges the order of the data, the data is inversely quantized by the inverse quantizer 40. Next, the inverse DCT unit 50 performs an inverse discrete cosine transform. The motion-compensated image reproduction unit 60 performs reproduction in consideration of the motion of the image. In MPEG2, a temporally intermediate frame (here, a B frame) is predicted from both a temporally earlier frame (here, an I frame) and a temporally later frame (here, a P frame). for that reason,
To reproduce the B frame, it is necessary to read out the predicted frame data 83 and 84 of the I frame and the P frame which have been decoded in advance from the frame memories 72 and 73 (MPEG).
In 2, the later P frame is decoded prior to the B frame). In the prediction method, there is a field prediction in addition to the above-described frame prediction. Similar to the frame prediction, the field prediction is performed based on a temporally earlier field (here, an I field) and a temporally later field (here, a field). A temporally intermediate field (here, the B field) is predicted from both the P field and the P field. Predicted frame data or predicted field data 83, 84 and inverse DC
The B-frame or B-field is reproduced by the motion-compensated image reproducing unit 60 based on the prediction error output from the T unit 50, and is written into the frame memory 74 as reproduced pixel data 85. I, P, in the frame memories 72, 73, 74
The B frame is read from each memory in a predetermined order (in FIG. 4, B frame image data 86 is read),
A reproduced image 200 is output.

The display processing means B shown in FIG. 3 performs processing such as noise removal and frame rate conversion of the reproduced image 200.

The conversion of the frame rate will be described below. For example, a movie is composed of 24 frames per second. This is encoded, decoded again, and PAL, SEC
In order to obtain a reproduced image of 25 frames / second (50 fields / second) like AM, the following two methods are conceivable.

The first method is a method on the premise that a reproduced image 200 of 25 frames / sec is obtained at the time of encoding. This is not always common, but MPEG2 /
This is possible on video stream syntax. At the time of decoding, one frame is decoded once every 1/2 second over a period of three fields, and in the reproduction image output, one frame is displayed only at that time, and during the other periods, two fields are displayed as usual. The start of decoding is set at irregular intervals, and the switching between two-field display and three-field display is performed by repeating the first field (re
peat_first_field).

In the second method, the frame rate is not converted at the time of encoding, the encoding is performed at 24 frames / sec, and the reproduced image 200 is obtained by decoding as it is. The decoding start time of each frame is at regular intervals, and the information of the repeat first field is ignored. In this case, the frame rate must be converted by the display processing means B in order to obtain an image of 25 frames / sec. The general method is to display the previous frame written on the second surface while writing on the first surface at 24 frames / sec (progressive) using two or more frame memories. Therefore, data is read out at 25 frames / second (interlace). At this time, it is necessary to read and adjust the same field twice every 1/2 second as in the first method so that the reading process does not overtake the writing process.

[0012]

In the above-mentioned first method of the prior art, the frame rate of the display monitor is fixed in advance for PAL / SECAM at the time of encoding.
The degree of freedom in selecting a display monitor is lost, and a decoding device connected to a monitor of NTSC or a personal computer cannot view an image even if it can decode a bit stream.

Further, the second method requires a frame rate conversion device having two or more expensive frame memories mounted outside the decoding processing means.

An object of the present invention is to provide an image decoding apparatus which solves the above-mentioned drawbacks of the conventional apparatus and can generate a reproduced image adapted to a display monitor connected on the decoding processing side and at a low cost. With the goal.

[0015]

According to the image decoding apparatus of the present invention, the input coded stream is, for example, 24 frames / sec and the display output is for 25 frames / sec. determines whether the number of input frames is in the low relationship than the number of output frames, the conversion mode holding means for holding the judgment result when in the low relationship, conversion mode
When the mode is set, the frame memory state determining means for determining whether or not the decoded image data to be displayed next is already stored in the frame memory ;
Image data for the frame to be stored in the frame memory
If not, the buffer memory state determining means for determining whether or not the undecoded image data to be displayed next is already stored in the buffer; and if the next image is stored in the frame memory or the buffer memory, If there is no data necessary for display, a re-display means for re-displaying the image data displayed immediately before is provided.

[0016]

According to the present invention, when a predetermined value is set in the mode holding means, it is determined whether or not the image data of the next frame to be displayed is complete based on the determination result of the frame memory state determining means. If not, check the output of the buffer memory state determination means.
If the undecoded image data of the frame to be displayed next is in the buffer memory, and the reference image data necessary to decode it is also present in the frame memory, the decoding / display processing is immediately performed. If not, the frame immediately before the currently displayed frame is redisplayed by the redisplay means. Accordingly, simple frame rate conversion can be performed without newly providing an external frame rate conversion device.

[0017]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described with reference to FIGS. FIG. 1 is a block diagram showing the configuration of one embodiment of the present invention. FIG. 2 schematically shows the timing of the process of decoding, reproducing and displaying an input bit stream of about 1 second according to the present embodiment.

In this embodiment, the input bit stream is encoded with a progressive sequence of 24 frames / sec, the display monitor assumes 25 frames / sec interlace, and the GOP (Group of Pictures) structure is , BIBBP (two B pictures exist between I or P pictures, and six pictures exist from an I picture to the next I picture).

In FIG. 1, the same components as those of the conventional example described with reference to FIG. 4 are given the same names and numbers. The components newly added from FIG. 4 include a mode register 61 in the motion-compensated image reproduction unit 60, a display clock generation unit 6
2, four signal lines of a vector decoding unit 63, a frame memory read / write control unit 64, an encoded display mode signal 11, a buffer memory status signal 12, a 24:25 conversion mode flag 65, and a display frame cycle signal 66.

On the other hand, in FIG. 2, an input bit stream 100 is input data to the buffer control unit 10 shown in FIG. Further, the decoding process is defined as the buffer memory 7
1 from the buffer memory 71 to the display monitor frame cycle (1/25).
Second), the data is read out frame by frame, and the variable length decoder 20, the scan converter 30, the inverse quantizer 40, and the inverse D
The processing time until the reproduction data is written to the frame memories 72, 73, and 74 via the CT unit 50 and the motion compensation image reproduction unit 60 is shown.

The writing of the frame memory in FIG.
The reading is performed by the processing contents of the frame memory read / write control unit 64 in the corresponding frame period, that is, writing of a reproduced image, reference reading, and reading for each of I, P, and B pictures.
The display read sequence is expressed using arrows. An arrow in the direction of entering each memory indicates writing of a reproduced image, an arrow in the vertical direction indicates reading as a reference image, and an arrow in the horizontal direction indicates reading for display. The reproduced image 200 is an output image obtained as a result of the above-described processing.

Next, the operation of this embodiment will be described with reference to FIGS. In FIG. 1, an input bit stream 1
00 is continuously stored in the buffer memory 71 via the buffer control unit 10 at a rate of 24 frames per second. However, since the coding method is different for each of the I, P, and B pictures, the generated code amount is different. The buffer control unit 10 analyzes the frame rate, the frame structure, whether or not the stream is a progressive sequence included in the stream, and notifies the mode register 61 via the encoded display mode signal 11. The buffer control unit 10 also has a function as memory information determination means, counts a picture start code indicating the beginning of each frame of coded data that enters and exits, and stores the number of coded frames in the current buffer memory 71. It monitors whether data is stored and notifies the result to the frame memory read / write control unit 64 via the buffer memory status signal 12. On the other hand, in the motion-compensated image reproducing unit 60, the input is 24 frames / sec progressive display and the display output is 25 frames such as PAL, based on the information from the coded display mode signal 11 and the scanning frequency of the currently connected display monitor. / Sec, the 24:25 conversion mode is set in the mode register 61 functioning as the mode holding means, and the result is set as the 24:25 conversion mode flag 65 as the buffer control unit 10 and the frame memory read / write control unit (frame memory status). (Functioning as a determination unit and a re-display unit). In this mode, the buffer control unit 10 controls the VBV delay (Video Buffering Verifier D) that specifies the read timing from the buffer memory 71 described in the stream.
elay) or DTS (Decoding Time Stamp) value, and is referred to only at the start of program reception, for example. Thereafter, the data is encoded in units of one frame from the buffer memory 71 in accordance with the display frame period signal 66. Read to decrypt the encrypted data. Normally, the read coded data is subjected to processing such as variable-length decoding, inverse quantization, inverse DCT, and motion picture compensation in real time, and each is determined by the type of picture (I, P, B). It is written to the frame memory. However, the input is 24
Since the output is 25 frames / sec while the output is 25 frames / sec, decoding pauses at a rate of one frame per second. This is the period during which the decoding process is indicated by x in FIG. At this time, the frame memory read / write controller 6
No. 4 controls output of a reproduced image according to the following priorities.

(1) First, if data for one frame in the order to be displayed is prepared in the frame memories 72 to 74, it is displayed. (2) Next, the buffer memory 71
If the encoded frame data (usually a B picture) in the order to be displayed is arranged for one frame, and the reference images necessary for decoding the encoded frame data are arranged in the frame memories 72 and 73, they are displayed while being decoded. (3) The above (1) and (2)
If both are not satisfied, the previously displayed frame is redisplayed.

As shown in FIG. 2, the buffer memory 71
If the encoded data for one frame has not been stored yet, the decoding process is paused, and the frame memory read / write control unit 64 determines that the reproduced image data to be displayed next is already in the I or P frame. First, it is checked whether the data is stored in the memories 72 and 73 (since only the I or P picture is decoded in advance and already stored in the frame memory as a reference image). If there is, the frame is output as a reproduced image 200. At this time, if it does not exist in the frame memories 72 and 73, the frame memory read / write control unit 64 outputs the immediately preceding frame for display again. Further, during a frame period after the decoding is paused once, the buffer control unit 10 confirms that the coded data for one frame has been prepared in the buffer memory 71, and decodes the frame. If the decoded data is a B picture, the frame memory read / write control unit 64 immediately outputs it as a reproduced image 200. On the other hand, if it is not a B picture, the data of the frame displayed immediately before is re-output.

In this embodiment, an input stream of 24 frames / second is converted to 25 frames / second such as PAL / SECAM.
Although the case of displaying and outputting in seconds has been described, if the frame rate of the input data is lower than the output frame rate by about several percent or less, for example, it can be applied to a device having a unique display frame rate such as a personal computer or a communication terminal. There is no practical problem. Although the GOP structure is described in this embodiment as BIBBP, it is easily understood by those skilled in the art that similar effects can be obtained with other combinations.

[0026]

As described above, the image decoding apparatus according to the present invention determines whether or not the number of images per second in the input coded stream is smaller than the number of display output images, and And a conversion mode holding means for holding the determination result in the case where the conversion mode is set. If the conversion mode is set, the decoded image data which is currently displayed and which is to be displayed next is already stored in the frame memory. a frame memory status determining means for determining whether there next frame to be displayed
If image data is not available in the frame memory,
If, next to be decoded before the image data in the buffer memory display, and a buffer memory status determining means for determining whether the reference picture data necessary for decoding the picture data is already stored in the frame memory And the change
In the switching mode, if the data necessary for displaying the next image is not stored in both the frame memory and the buffer memory according to the determination results of the frame memory state determining means and the buffer memory state determining means, the data is displayed immediately before. Since it has a re-display means for re-displaying image data, for example, the same frame is re-displayed at the time of an underflow by a frame memory control and a buffer control, and the PAL is directly converted to a PAL which has been encoded in a progressive sequence of 24 frames / sec. Since it can be output as a reproduced image of 25 frames / second in a system or the like, it is not necessary to specify a display monitor in advance at the time of encoding or to add an expensive frame rate conversion device or the like to the outside. The increase can be kept to a minimum.

Also, the input is not 24 frames / sec but MP
Even in the case of 23.976 frames / sec recognized by EG2, conversion can be performed without any modification.

[Brief description of the drawings]

FIG. 1 is a block diagram showing the configuration of an embodiment of the present invention.

FIG. 2 is a diagram showing a schematic timing of a process of decoding, reproducing and displaying an input bit stream for about one second in one embodiment of the present invention.

FIG. 3 is a block diagram illustrating a configuration example of a conventional image decoding device.

FIG. 4 is a block diagram showing details of a configuration of a decoding processing unit in FIG. 3;

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 10 Buffer control part 11 Encoding display mode signal 12 Buffer memory status signal 20 Variable length decoder 30 Scan converter 40 Dequantizer 50 Inverse DCT part 60 Motion compensation image reproduction part 61 Mode register 62 Display clock generation part 63 Vector decoding Unit 64 frame memory read / write control unit 65 24:25 conversion mode flag 66 display frame period signal 71 buffer memory 72 frame memory 73 frame memory 74 frame memory 81 data 82 data 83 predicted field data 84 predicted field data 85 reproduced image data 86 B frame Image data 100 Input bit stream 200 Playback image

Continued on the front page (72) Inventor Tomoyuki Shindo 4-36-19 Yoyogi, Shibuya-ku, Tokyo Inside Graphics Communication Laboratories Inc. (72) Inventor Yutaka Okada 4-36, Yoyogi, Shibuya-ku, Tokyo No. 19 Inside Graphics Communication Laboratories Co., Ltd. (72) Inventor Norihiko Nagai 4-36-19 Yoyogi Shibuya-ku, Tokyo Inside Graphics Communication Laboratories Co., Ltd. (72 ) Inventor Ryuji Saito 4-36-19 Yoyogi, Shibuya-ku, Tokyo Inside Graphics Communication Laboratories (72) Inventor Yoshikazu Kawamura 4-36-19 Yoyogi, Shibuya-ku, Tokyo Stock Company (56) References JP-A-6-197273 (JP, A) JP-A-6-78279 (JP, A) (58) Fields studied (Int. Cl. ⁷⁾ , B name) H04N 7/24 - 7/68 H04N 7/01

Claims (1)

(57) [Claims] In an apparatus for decoding encoded image data, it is determined whether or not the number of images per second in an input encoded stream is smaller than the number of display output images. a conversion mode <br/> over de holding means for holding the judgment result when, if the conversion mode is set, the decoded image data to be displayed on the next currently displayed already stored in the frame memory Frame memory state determination means for determining whether or not the image data of the next frame to be displayed
If the image data is not present in the frame memory, it is determined whether or not the image data to be displayed next to be decoded is already stored in the buffer memory and the reference image data necessary for decoding the image data is already stored in the frame memory. A buffer memory state determining means for determining, and in the conversion mode, displaying the next image in both the frame memory and the buffer memory according to the determination results of the frame memory state determining means and the buffer memory state determining means. An image decoding apparatus with a frame rate conversion function, comprising: a re-display means for re-displaying the image data displayed immediately before, if no data is available.