JP2000134581A - Device and method for reproducing moving picture - Google Patents

Abstract

PROBLEM TO BE SOLVED: To prevent the influence of skipping from being given to another frame in the case one frame reproduction is skipped and to enable a moving picture reproduction processing to be executed, which is adapted to the throughput of a microcomputer, while suppressing the deterioration of image quality in moving picture reproduction by skipping the reproduction of the B-picture of the frame when a reproduction image is monitored and the delay of the processing is judged. SOLUTION: A decoding circuit 41 fetches the bit stream of the moving picture which is compressed by an MPEG: 1 and the decoding image of the MPEG image is reproduced and outputted by a decoding algorithm. A display controller 42 controls the operation of a display buffer 43 for fetching the decoding image from the decoding circuit 41 and also permits a skip signal 42a to be active in the case the fetching of the decoding image seems to be delayed after monitoring an operation state. Skip judging equipment 40 judges the frame to be skipped by a prescribed algorithm while the skip signal 42a is active and outputs the judgement result signal 40a to the decoding circuit 41.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a moving picture reproducing apparatus and a moving picture reproducing method, and more particularly to an MPEG (Moving Picture).
Coding Experts Group) Standard coding method (MPEG
The present invention relates to a moving image reproducing apparatus and a moving image reproducing method for reproducing a moving image compressed by 1 or MPEG2).

[0002]

2. Description of the Related Art MPEG1 and MPEG2 have a transfer speed (about 1.5 Mbit / sec for MPEG1 and several M to several tens Mbit / sec for MPEG2) and an application object (MPEG1 is mainly a storage medium such as a CD-ROM). Although MPEG2 has differences in storage media and broadcast / communication media such as next-generation television, etc., its compression technology uses DCT (Discrete Transform) and motion compensation (MC:
Motion Compensation) is common in that a so-called hybrid coding method combining inter-frame prediction is adopted.

[0003] DCT is an international standard for still image compression, which is also called discrete cosine transform, and is a method of encoding using the correlation between levels of adjacent pixels in an image. The transform coefficients are quantized and the quantization numbers are encoded. Since a still image has correlation in both the horizontal and vertical directions, two-dimensional conversion is generally used. Image data is m pixels x n
The block is divided into line blocks, and the blocks are orthogonally transformed and quantized.

On the other hand, in a moving image (moving image), the current frame (current frame) and the immediately preceding frame (previous frame) are often similar except for a scene change due to editing. For example, NTSC (National Televis
29.97 TV images of the Ion System Committee)
Frame / sec, depending on the content of the image, about 1/3
The change of the image during 0 seconds is not always large. Therefore, in moving image coding, a past frame is used for prediction of a current frame. However, the first frame (first frame) uses a still image coding method. In the coding of a still image, not only prediction coding and transform coding, but also data used in coding is closed in the image. No other image is used for prediction. For this reason, a method of encoding only from information in a frame is called intra-frame encoding or intra-coding, and a frame encoded by that method is referred to as an intra-coded picture (intra-coding picture). : "I" picture for short).

[0005] In moving picture coding, forward and bidirectional inter-frame prediction is used. The former is called inter-frame coding, and a frame coded by this method is called an inter-frame coded frame (predictive coding pictur).
e: Abbreviated as “P” picture). The latter is called frame interpolation coding, and a frame coded by this method is a frame interpolation coded frame (bidirectiona coding frame).
ly-coding picture (abbreviated as “B” picture).

FIG. 8 shows a GOP (Gr.
oup Of Pictures: random access unit of video) This is an example of a layer structure. In the figure, the part indicated by a diamond-shaped figure is a schematic representation of a frame, the alphabet in the figure represents the picture type, and the numbers represent the order of encoding (also the order of decoding). The first and last frames of a GOP are always I pictures, and any type of picture is arranged between them. For example, in the illustrated example, two B pictures and one P picture are alternately arranged on the time axis.

[0007] Here, as described above, the I picture is a closed frame having no reference frame separately, but the P picture and the B picture are frames encoded by referring to other frames. The direction of reference is indicated by the arrow line in the figure. That is, a P picture is a forward predicted frame using a past I picture (or P picture) as a reference frame, and a B picture is a past and a future (future is a convenient term that means a time later in the future). Yes; the same applies hereinafter)
This is a bidirectional prediction frame using the I picture (or P picture) as a reference frame.

More specifically, an I picture is obtained by performing a DCT operation on an original signal for each block, and then applying a DC component having a step size of 8 and an AC component to a value obtained by multiplying a quantization matrix by a constant called a quantization scale. The value is quantized as a size, and the value after quantization is coded by a variable length code. The P picture is a motion compensation inter-frame prediction error signal and a motion compensation signal between an already coded I picture or P picture. This is a coded vector. Also, the B picture is an encoded I picture or P picture that is temporally later, that is, a motion compensation prediction from the previous frame, a motion compensation prediction from the later frame, and a motion compensation prediction from both the previous and next frames. It is adaptively selected and coded for each macroblock.

[0009]

By the way, in the above-described moving picture coding, inter-frame prediction is performed on P-pictures and B-pictures. Therefore, when this coded picture is decoded (reproduced), all the frames are decoded. It must be regenerated without omission. For example, if the reproduction of P5 in FIG. 8 fails, the frames of B6 and B7 referring to P5 cannot be reproduced as well, and as shown in FIG.
This is because as a result of losing the first three frames (broken-line frames), frame skipping occurs and the image quality of a moving image is greatly impaired.

[0010] Such a problem is particularly serious for microcomputer application equipment having no dedicated moving picture processing circuit. This is because the microcomputer must perform various other processes simultaneously while performing the playback and display processes of the moving image, and unless a very high-performance microcomputer is used, the processing capacity is naturally limited. Because there is.

Therefore, the present invention adapts to the processing capability of a microcomputer while suppressing the influence of the skipping of one frame on other frames even if skipping the reproduction of one frame. The purpose is to cause a moving image reproduction process to be performed.

[0012]

According to a first aspect of the present invention, there is provided a moving picture reproducing apparatus for reproducing a coded moving picture composed of a large number of frames having any one of I picture, P picture and B picture types. A reproducing apparatus comprising: a determination unit that monitors a reproduced image to determine a processing delay; and a skip unit that skips reproduction of the B picture of the frame when the processing delay is determined by the determination unit. It is characterized by. A moving picture reproducing apparatus according to claim 2, wherein the reproduced picture is monitored in a moving picture reproducing apparatus for reproducing an encoded moving picture composed of a number of frames having any one of picture types of I picture, P picture and B picture. Determining means for determining a processing delay, and when the determining means determines a processing delay, a search is made for a frame located before the I picture in the frames, and a frame preceding the searched frame is determined. And a skip unit for skipping reproduction of the searched frame when the frame is not a B picture.
A moving picture reproducing apparatus according to claim 3, which monitors a reproduced picture in a moving picture reproducing apparatus for reproducing an encoded moving picture composed of a number of frames having any one of picture types of I picture, P picture and B picture. Determination means for determining a processing delay, and when the processing delay is determined by the determination means, if there is a B-picture frame in the frames, the reproduction of that frame is skipped. And searching means for searching for a frame located before the I-picture, and skipping playback of the searched-out frame when the frame preceding the searched-out frame is not a B-picture. I do. The moving image reproducing apparatus according to claim 4 is the method according to claim 1,
4. The moving image reproducing apparatus according to claim 2, wherein said skip means prevents skipped frames from continuing. A moving picture reproducing method according to claim 5, wherein the reproduced picture is monitored in a moving picture reproducing method for reproducing an encoded moving picture composed of a large number of frames having any one of picture types of I picture, P picture and B picture. And a second step of skipping the reproduction of the B picture of the frame when the processing delay is determined by the first step. 7. The moving picture reproducing method according to claim 6, wherein the reproduced picture is monitored in a moving picture reproducing method for reproducing an encoded moving picture composed of a number of frames having any one of picture types of I picture, P picture and B picture. First to determine the processing delay
And a step of searching for a frame located before the I-picture in the frame when the processing delay is determined in the first step, and when the frame before the searched frame is not a B-picture, And a second step of skipping reproduction of the searched frame. 8. The moving image reproducing method according to claim 7, wherein the reproduced image is monitored in a moving image reproducing method for reproducing an encoded moving image composed of a large number of frames having any one of I picture, P picture, and B picture types. A first step of determining a processing delay by the first step; if the processing delay is determined by the first step, if there is a B-picture frame in the frames, the reproduction of that frame is skipped; I in
When a frame located before a picture is searched, and a frame before the searched frame is not a B picture,
And a second step of skipping the reproduction of the searched frame. In a moving image reproducing method according to an eighth aspect of the present invention, in the moving image reproducing method according to the fifth, sixth or seventh aspect, the second step means prevents skipped frames from continuing. The recording medium according to claim 9 is the recording medium according to claim 1 or claim 2.
Alternatively, a program for realizing the determination means and the skip means according to claim 3 is stored.

[0013]

Embodiments of the present invention will be described below with reference to the drawings. In FIG. 1, reference numeral 40 denotes a skip determiner (skip means), 41 denotes a decoding circuit, 42 denotes a display controller (judgment means), and 43 denotes a display buffer.
The decoding circuit 41 outputs a moving image (MP
An EG image) is fetched, and a decoded image of an MPEG image is reproduced and output based on a known decoding algorithm. The display controller 42 controls the operation of the display buffer 43 that takes in the decoded image from the decoding circuit 41, and monitors the operation state of the display buffer 43, and activates the skip signal 42a when it becomes too late to take in the decoded image. . The skip determiner 40 determines a frame to be skipped based on a predetermined determination algorithm while the skip signal 42a is active, and outputs a determination result signal 40a to the decoding circuit 41.

FIG. 2 is a flowchart of the judgment algorithm. This determination algorithm does not affect the creation of another frame among the frames constituting the GOP layer, in other words, finds a frame that is not a reference frame of another frame, and is based on the following facts. It is. <Fact 1> As described at the beginning, the frame constituting the GOP layer has a picture type from any one of I, B, and P pictures, and the B picture among them is the prediction source (reference frame). In fact, even if a B picture frame is skipped (frame expansion processing is not performed), the creation of other frames is not affected at all. <Fact 2> Regardless of the picture type of the own frame, even when the next frame on the time axis is an I picture and the previous frame is not a B picture, the own frame is skipped (frame expansion processing is performed. No) has no effect on the creation of other frames.

In FIG. 2, steps S1 to S4 correspond to the current frame (own frame) based on the above two facts.
This is a skip flow. That is, when the picture type of the current frame is “B” (YES determination in S1), or when the picture type of the current frame is other than “B” (NO determination in S1), and the picture type of the next frame on the time axis, Is "I" (YES determination in S3) and the picture type of the previous frame is not "B" (NO determination in S4), the current frame is skipped (S2).
In either case, based on Fact 1 or Fact 2, skipping the current frame has no effect on the creation of other frames.

Here, a comparison will be made between the skip processing of the present embodiment and the frame drop of the conventional example. FIG. 3 is a comparison table. In FIG. 3, crosses indicate skip processing and dropped frames. In the conventional example, for example, P of # 7
If the processing of the picture was not in time, this # 7
B pictures # 5 and # 6 referencing the P picture
10 P-pictures dropped a frame, and this # 10
B pictures # 8 and # 9 referencing the P picture
Thirteen P pictures dropped in the frame, and this # 1
The B pictures # 11 and # 12 and the B pictures # 14 and # 15 that refer to the P picture No. 3 are dropped, resulting in one frame (P picture # 7).
The processing of # 5 to # 1 was incidentally just because the processing of
Because you lose 11 frames up to 5,
Significant degradation in image quality is evident.

On the other hand, in this embodiment, when the processing of a frame is about to be delayed, if the picture type of the current frame is "B", the frame is unconditionally skipped. The crosses of # 8, # 9, and # 14 shown in the figure are B picture frames, which are frames that have been intentionally skipped. As is apparent from the figure, according to the present embodiment, there is no spillover of dropped frames other than intentionally skipped frames. Therefore, it is possible to reduce the number of frames to a minimum suitable for the capability of the moving image reproduction process, and to avoid a significant deterioration in image quality.

The other steps (S5 to S9) in the judgment algorithm shown in FIG. 2 are countermeasures in a case where there is no appropriate frame that corresponds to the above two facts. That is, when the processing of a frame is about to be missed, if the current frame at that time falls under any of the above two facts, the processing delay can be recovered by skipping that frame. ,
If the current frame does not apply to any of the above two facts, the frame will be dropped as in the conventional example unless some care is taken, and the image quality will be greatly degraded.

The first countermeasure is that if there is a B picture for which no skip is specified between the next I picture, the B picture
This is to skip the picture (S5, S5
6). The second measure is to skip the P picture if there is a P picture closest to the I picture that has not been designated for skipping until the next I picture (S7, S8). The third measure is to skip all frames from the current frame to the next I picture (S9).

If the current frame does not apply to either of the above two facts, the measures are evaluated sequentially from first to third. The first countermeasure is skipping only B pictures and has no effect on other frames, and the second countermeasure is skipping only P pictures under specific conditions and has little effect on other frames ( At least there is no ripple to other P pictures). The third measure is to skip all frames after the current frame. This is the worst case when the first and second measures are passed. Such a worst case will not fall unless the processing power is very low.

In consideration of smoothness of reproduction of a moving image, it is desirable not to skip frames continuously. FIG. 4 shows the improvement flow, which is applied to the skip processing (S2, S6, S8, S9) in FIG. That is, when the frame immediately before the frame specified to be skipped is not skipped (S10), the frame specified to be skipped is skipped (S11).
In other words, if the frame immediately before the skip-designated frame has already been skipped, the skip designation is cancelled. According to this, continuous skipping of frames is avoided, so that smoothness of moving image reproduction is not lost.

FIG. 5 shows a decision algorithm improved so that frame skips are made at equal intervals as much as possible.
In this algorithm, first, a frame that has not been designated for skipping up to the next I picture and that does not affect the reproduction of other frames even if the skip is performed is searched (S20). A frame at the position (*) closest to the interval of the previously skipped frame among the frames is designated to be skipped (S2).
2). Note that the other steps (S21 to S25) are countermeasures when the corresponding frame is not found (NO determination in S21), and this countermeasure has not been specified as a skip until the next I picture. , If there is a P picture closest to the I picture, the P picture is skipped; otherwise, all frames from the current frame up to but before the next I picture are skipped.

If the frame number of a frame skipped two times before (serial number sequentially assigned to each frame along the time axis from the beginning of the sequence) is a, and the frame number of the previously skipped frame is b, b +
The (ba) -th frame is the frame closest to the interval of the previously skipped frame. FIG.
= 2, b = 5. The frame number skipped this time is 5+ (5-2) = 8, that is, # 8
And skip frames (# 2, # 5, #
It can be seen that 8) are arranged at equal intervals every third. Therefore, according to the present embodiment, as shown in FIG. 7, the skip frames are improved so as to be at equal intervals as much as possible, so that unnatural frame drop is suppressed and smoother moving image reproduction is enabled. Can be.

In the present embodiment, the skip frame number of the current time is determined using the skip frame numbers up to two before to put the skip frames at equal intervals, but the present invention is not limited to this. For example, the previous skip frame interval may be stored and used directly, or longer-term information may be collected and averaged.

Further, in each of the above embodiments, the search range of the frame to be skipped is set to the next I picture, that is, within one GOP layer.
The present invention is not limited to this, and it may be within a plurality of continuous GOP layers.

[0026]

According to the first or fifth aspect of the present invention, when a reproduction image is monitored and a processing delay is determined,
Since the reproduction of the B picture is skipped, the B picture is a frame having no influence on the reproduction of the other frame. Therefore, it is possible to adjust the frame according to the processing capability without losing the other frame incidentally. . According to the second or sixth aspect of the present invention, when a delay in processing is determined by monitoring a reproduced image, a frame under a predetermined condition (a frame located before an I picture and a frame before that frame) The frame of the predetermined condition is not affected by the reproduction of the other frames as in the case of the B picture, so that the other frames are incidentally lost. Not
Frame adjustment according to processing capacity can be performed. According to the third or seventh aspect of the present invention, when the reproduction image is monitored and the processing delay is determined, first, if there is a B picture frame, the reproduction of that frame is skipped. Since the reproduction of the condition frame (the frame located before the I picture and the frame preceding the frame is not a B picture) is skipped, it can be applied to an image in which various picture types are mixed. According to the fourth or eighth aspect of the present invention, in the moving image reproducing apparatus according to the first, second or third aspect, the skip means prevents the frames to be skipped from being continuous. The accompanying unnaturalness of the image can be suppressed, and a smooth moving image can be obtained.

[Brief description of the drawings]

FIG. 1 is a block diagram of an embodiment.

FIG. 2 is a flowchart of a determination algorithm according to the first embodiment.

FIG. 3 is a skip comparison table of a conventional example and the first embodiment.

FIG. 4 is an improved flowchart of a determination algorithm according to the first embodiment.

FIG. 5 is a flowchart of a determination algorithm according to the second embodiment.

FIG. 6 is a conceptual diagram of the specification of an equal interval of a skip frame.

FIG. 7 is a skip table according to the second embodiment;

FIG. 8 is an example of a structure of a GOP layer.

FIG. 9 is a conceptual diagram of a frame drop in a conventional example.

[Explanation of symbols]

 40 skip determiner (skip means) 42 display controller (judgment means)

 ──────────────────────────────────────────────────続 き Continued on the front page F term (reference) 5C053 FA30 GA11 GB22 GB26 GB32 GB37 GB38 HA29 HA40 LA11 5C059 KK01 KK21 MA00 MA23 MC11 MC38 ME01 PP05 PP06 PP07 SS11 SS20 TA07 TB04 TC36 TD01 UA02 UA05 5C077 MP08 NP02 PP43 RR21

Claims (9)

[Claims] 1. A moving image reproducing apparatus for reproducing an encoded moving image composed of a large number of frames having any one of I picture, P picture, and B picture. A moving image reproducing apparatus, comprising: a judging means; and a skip means for skipping reproduction of a B picture of the frame when the processing delay is judged by the judging means. 2. A moving image reproducing apparatus for reproducing an encoded moving image composed of a large number of frames having any of picture types of I picture, P picture and B picture. A determining means for determining, when the processing delay is determined by the determining means, searching for a frame located before the I picture in the frame, and when the frame before the searched frame is not a B picture, And a skip means for skipping the reproduction of the searched frame. 3. A moving image reproducing apparatus for reproducing an encoded moving image composed of a large number of frames having any one of I picture, P picture and B picture, monitors the reproduced image to reduce the processing delay. Determining means for determining, and when a delay of processing is determined by the determining means, if there is a B-picture frame in the frames, the reproduction of that frame is skipped. And a skip unit for skipping reproduction of the searched frame when a frame located before the searched frame is not a B picture. 4. The moving picture reproducing apparatus according to claim 1, wherein said skip means prevents skipped frames from continuing. 5. In a moving image reproducing method for reproducing an encoded moving image composed of a number of frames having any one of I picture, P picture, and B picture, a reproduced image is monitored to reduce a processing delay. A first step of determining; and a second step of skipping the reproduction of the B picture of the frame when the processing delay is determined by the first step.
A moving image playback method, comprising: 6. A moving image reproducing method for reproducing an encoded moving image composed of a large number of frames having any of picture types of I picture, P picture and B picture. A first step of determining, and when a delay of processing is determined by the first step, a frame located before an I picture is searched for in the frame, and a frame preceding the searched frame is B
A second step of skipping the reproduction of the searched frame when the frame is not a picture. 7. A moving image reproducing method for reproducing an encoded moving image composed of a number of frames having any of picture types of I picture, P picture, and B picture. A first step of determining, and when a delay in processing is determined by the first step, if there is a B-picture frame in the frames, the reproduction of that frame is skipped; Retrieving a frame located before the searched frame, and skipping the reproduction of the retrieved frame when the frame preceding the retrieved frame is not a B picture. Method. 8. The moving picture reproducing method according to claim 5, wherein said second step means prevents skipped frames from continuing. 9. A recording medium storing a program for realizing the determination means and the skip means according to claim 1, 2, or 3.