JPH08130707A - Picture signal processor - Google Patents

Abstract

PURPOSE: To perform smooth reverse direction reproduction with relatively small memory capacity by preparing only a memory for storing I and P frames. CONSTITUTION: Encoding is performed in the order of frames 0(I), 3(P), 1(B), 2(B), 6(P), 4(B), 5(B)... and the sequence is rearranged to the order of the frames 27(I), 25(B), 9(I), 23(B), 22(B), 12(P)... in a buffer. Then, a memory group is constituted of reference frame memories (A-D) equivalent to four screens and an output memory equivalent to one screen, and at the time of using both front prediction and back prediction and reproducing movement compensation type compressed video signals in a reverse direction, the frames (I) and (P) to be referred to from the other frames are decoded in advance, restored pictures are tentatively stored and the frames (B) not to be referred to from the other frames are decoded in the order of reproduction. Thus, just by preparing the memory for storing the frames (I) and (P), the smooth reverse direction reproduction is realized.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image signal processing device for reproducing a digital moving image recorded in a compressed state, and more particularly to an image signal processing device having a low speed backward reproduction function.

[0002]

2. Description of the Related Art MPEG (Moving Picture Experts Gro
In moving image compression by MPEG1 and MPEG2, which is being standardized by up), the screen (frame) is divided into small blocks (macroblock = rectangle of 16 pixels × 16 pixels) and Extracts a region (reference image region) similar to the macroblock to be encoded, and calculates the spatial distance and orientation (motion vector) from the reference image region and the difference information between the reference image region and the region to be encoded. However, these pieces of information are compressed using DCT and variable length coding. Compressing only the motion vector and the difference information is much more efficient than compressing the original image itself.

However, the frame compressed by the difference information cannot be restored without a reference frame. Therefore, in order to realize variable speed reproduction and random access using a medium on which such information is recorded, periodically,
It is necessary to provide a frame that does not refer to other images. This frame is called an I (Intra) frame. The next image is compressed using the I frame as a reference frame, and the next image is further compressed using the already compressed image as a reference frame. Among frames to be compressed using the reference image, only the frame preceding in time is used as the reference image P
(Predictive) frame and B (Bidirectionally-predictive) in which temporally preceding and following frames are used as reference frames
There is a frame.

FIG. 6 shows an example of the interframe prediction method.

In this figure, the input image is encoded as an I frame at a cycle of 9 frames, is encoded as a P frame at a cycle of 3 frames, and the rest is encoded as a B frame.

First, frame 0 is an I frame.
It is encoded without reference to other images, then frame 3 is encoded as a P frame with reference to the previous frame 0. Next, the frames 1 and 2 are encoded as B frames with reference to the preceding and following frames 0 and 3. That is, the encoding is performed in an order different from the original frame order. The subsequent steps are similarly encoded. This order is correctly restored on decoding.

The combination ratio of I, P and B can be set arbitrarily. By combining I, P, and B in an appropriate ratio, it is possible to obtain a compressed image signal with high compression efficiency and random access.

When restoring a compressed signal using bidirectional prediction as in this example, usually, two frames of reference frame memory and one frame of output image memory are prepared for decoding. An outline of normal decoding processing will be described with reference to FIG.

FIG. 7 illustrates a decoder processing order and a memory occupation order. Reference characters A and B denote image memories which are reference memories. As described above, in the example of FIG. 6, frames 0 (I), 3 (P), 1 (B), 2 (B), 6
(P), 4 (B), 5 (B)-are encoded in this order. Therefore, the decoder also decodes in this order.

First, the frame 0 (I) is decoded and stored in the image memory A. Next, using this image as a reference frame, the frame 3 (P) is decoded and stored in the image memory B. Next, the images in the image memories A and B are used as reference frames, and frames 1 (B) and 2 (B) are used.
Is decoded and temporarily stored in the output memory to produce an interlaced screen, and then output. In the output memory, the result decoded in macro block units is converted in the order of normal scanning lines and output in accordance with a predetermined output timing.
Before outputting the frames 1 (B) and 2 (B), the frame 0 (I) is output to the output memory. Next, the frame 3 (P) stored in the memory B is output to the output memory. Next, the frame 3 (P) of the memory B is used as a reference frame, and the frame 6 (P) is decoded and stored in the image memory A. In this way, by sequentially using the memory to decode, by preparing reference memories for two screens, it is possible to restore the compressed signal using bidirectional prediction.

When variable speed reproduction is performed using the reproduction circuit having such a structure, in the case of forward reproduction, if the processing speed is changed according to the reproduction speed, reproduction can be performed by the same procedure as normal speed reproduction. is there. For example, if the double speed reproduction is performed, the operating speed of the circuit may be doubled and the obtained output image may be appropriately dropped.

However, in the reverse reproduction, the following problems occur. In FIG. 6, reverse reproduction is performed in the order of frames 9, 8, and 7. First, since the frame 9 is an I frame, it can be reproduced without any problem. Next, in order to reproduce the frame 8, it is necessary to use the frame 6 as a reference frame, but in order to obtain the image of the frame 6,
Frame 0 and frame 3 must be decoded. That is, the reference frames for the frames 7 and 8 can be obtained only after decoding the frames 0, 3, and 6 one after another. Moreover, since the image memory for the reference frame has only two screens, the decoded images of the frames 0 and 3 cannot be stored. Therefore, in order to decode the frames 4 and 5, next, the frames 0 and 3 are newly read.
Need to be decoded. As described above, since it is necessary to decode the same frame many times, it is practically impossible to perform the reproduction processing at the speed several times as fast as the reproduction speed when performing the backward reproduction.

In order to solve this problem, in the example of FIG. 6, a memory for storing decoded images for 9 frames is prepared, and frames 0 to 8 are decoded in the forward direction and all are stored in the memory. , There is a method of realizing reverse reproduction by reading in the reverse direction.

Further, as disclosed in Japanese Patent Laid-Open No. 5-1553577, images to be reproduced are selected and only I frames or I and P frames are reproduced to realize backward reproduction without increasing the processing speed. There is also an example.

[0015]

However, the former requires a huge memory. Further, in MPEG1 and MPEG2, the cycle of the I frame is not defined, so the memory is prepared assuming the cycle of the I frame in advance. Will not be able to respond. Further, in the example of FIG. 6, it takes time to decode the data of 9 frames from the reception of the reverse reproduction instruction to the output, which is not practical.

In the latter case, in the example of FIG. 6, an image can be restored only every 9 frames when only I frames are selected and every 3 frames when I and P frames are selected. In high-speed reverse reproduction, there is no problem with such thinned-out reproduction, but in low-speed reverse reproduction such as -1 × speed, the reproduced image is out of frames and is unsightly.

It is an object of the present invention to provide an image processing apparatus capable of executing a motion-compensated compressed video signal in the reverse direction by using both forward prediction and backward prediction without increasing the memory capacity. To provide.

[0018]

An image signal processing apparatus according to the present invention uses reproduced data for a frame (I, P frame) referred to by another frame and a frame (B frame) not referred to by another image. ), First I, P
The frames are decoded in the same order as in normal reproduction and stored in the memory for recording reference frames, and the B frames are decoded in the order in which the reference images are prepared and in the order in which they are output.
Reverse reproduction is realized by outputting in a predetermined order.

The reference frame recording memory may be prepared in advance with a sufficient capacity for realizing the backward reproduction. However, in the backward reproduction, a slight deterioration of the image quality is allowed, and therefore, in the screen. Reduce the data using processing such as decimation,
It is also possible to store all the reference images required for reverse reproduction in the memory for two screens prepared for normal speed reproduction. When the cycle of the I frame of the input signal is longer than the cycle assumed in advance, the reference image is compressed in the screen or decimated in the screen so that the required reference image can be stored in the prepared memory.

[0020]

According to the present invention, smooth backward reproduction can be realized with a relatively small memory capacity simply by preparing a memory for storing I and P frames.

In particular, by using thinning together, smooth reverse reproduction can be realized with the same memory capacity as normal speed reproduction. In addition, I that is longer than the expected I frame period
Even in the case of a reproduction signal having a frame period, smooth reverse reproduction is performed.

The increase in delay during reverse reproduction is I, P
Since the time required to decode the frame is only required, the backward playback image is output promptly after receiving the backward playback instruction.

[0023]

1 is a block diagram of an embodiment in which the present invention is applied to a digital VTR, a digital video disk and the like. The configuration of the embodiment of FIG. 1 will be described below together with the operation.

The signal recorded on the recording medium 1 is reproduced by the reproducing head 2 while adjusting the reproducing condition by using the servo circuit 3 according to the reproducing direction and the reproducing speed, and a predetermined reproducing signal processing is performed. It is possible to provide a plurality of reproducing heads 2 if necessary. The reproduced signals are rearranged in the buffer 11 in a predetermined data order. Depending on the recording format and the processing of the servo circuit 3, it is possible to perform the reproduction in which the rearrangement has been completed to some extent at the stage of being reproduced by the reproducing head 2. In this case, buffer 1
The rearrangement by 1 is unnecessary, or the target rearrangement can be realized with a smaller rearrangement amount. The rearrangement order differs depending on the reproduction direction and the reproduction speed.

The rearranged signals are sent to the variable length decoder 1
Variable length decoding is performed at 2. The decrypted data is DC
It is composed of T data, motion vectors, and the like. These data are separated by the data separation circuit 13, inversely quantized, quantized by the IDCT circuit 14, and restored to data before DCT. Of the restored data, the I-frame data does not refer to other images, so the memory group 16 does not change.
To be stored. Since the difference between the P and B frame data and the reference image is encoded, first, the memory group 16
A predetermined reference frame is selected from the
At, the motion compensation is performed using the motion vector in the reproduced data to generate the reference image, and the decoded difference signal and the reference image are added by the adder 15 to restore the image, and the memory group 1
6 is stored. The memory group 16 outputs a predetermined frame corresponding to the reproduction speed and direction to the output terminal 17.

FIG. 2 illustrates the processing order of the decoder and the memory occupying order when the backward reproduction (minus 1 × speed) is performed in this embodiment. In this example, the memory group 16
Reference frame memory for 4 screens (A, B, C, D)
And an output memory for one screen, so that an image in the sequence shown in FIG. 6 can be reproduced in the reverse direction.

In this sequence, as described with reference to FIGS. 6 and 7, frames 0 (I), 3 (P), 1 (B), 2
(B), 6 (P), 4 (B), 5 (B)-are encoded in this order. This sequence in buffer 11
As shown in FIG. 2, frames 27 (I), 18 (I),
21 (P), 24 (P), 26 (B), 25 (B), 9
(I), 23 (B), 22 (B), 12 (P) --- are rearranged in this order. In this order, the I and P frames are in the same order as the forward direction so that the reference relation is maintained, and the B frames are the frames in which the decoding of the reference frame is completed are arranged in the reverse direction (output order). It is a thing. This rearrangement is performed not only in the buffer 11 but also in the recording medium 1.
It is also possible to use the servo circuit 3 or the like at the stage of reproducing from. Further, the capacity of the buffer 11 can be reduced by using both in combination.

First, the frames 27 (I) and 18 (I) are decoded and stored in the memory D and the memory A. Next, the frame 18 (I) is used as a reference frame, and the frame 21 (P) is decoded and stored in the memory B. Similarly, using frame 21 (P) as a reference frame,
The frame 24 (P) is decoded and stored in the memory C. At this point, since the reference screens for decoding the frames 26 (B) and 25 (B) are prepared, first, the image of the frame 27 (I) is output to the output memory, and then the frame 2
Frame 2 with 7 (I) and 24 (P) as reference frames
6 (B) and 25 (B) are decoded and output to the output memory. At this point, the data of frame 27 (I) is no longer needed, so frame 9 (I) is decoded and stored in memory D. Next, the image of the frame 24 (P) is output to the output memory, the frames 24 (P) and 21 (P) are used as reference frames, and the frames 23 (B) and 22 (B) are decoded and output to the output memory. . At this point, the data of frame 24 (P) is no longer necessary, so frame 1
2 (P) is decoded and stored in the memory C. By thus using the memories A to D, the sequence of FIG. 1 can be reproduced in the reverse direction without increasing the processing speed. Further, in the case of performing reverse reproduction at a slower speed, it suffices to adjust the cycle of reading from the output memory and output an appropriate frame twice or more.

FIG. 3 shows the processing order of the decoder and the memory occupying order when the minus double speed is realized using this embodiment. Since the I and P frames are reference frames for other images, the decoding cannot be omitted. However, since the B frame is not a reference image for other images, the image skipped by minus 2 times reproduction is Decoding can be omitted. Therefore, in the buffer 11, unnecessary B frame data (an odd B frame in FIG. 3) is deleted and rearranged. afterwards,
The I and P frame data and the necessary B frame data are decoded by the procedure described with reference to FIG.
Output from output memory at appropriate timing. With such an operation, the amount of data to be decoded is reduced, so that it is possible to minimize the increase in processing speed at high speed and double speed.

FIG. 4 is a block diagram of another embodiment in which the present invention is applied to a digital VTR, a digital video disc and the like.

In this embodiment, in order to realize backward reproduction with a small amount of reference image memory, the reference image is reduced in advance and stored in the memory.

In FIG. 4, parts 1 to 18 operate in the same manner as the embodiment of FIG. 1 and decode the reproduced signal. The reproduced image is reduced by the reduction circuit 21 to reduce data,
It is stored in the memory group 16. In this embodiment, the memory group 16
Is composed of a reference image memory for two screens and an output memory for one screen, as in the conventional reproducing apparatus.
The output image signal is enlarged by the enlargement circuit 22 to the original size and is outputted. Reference numeral 23 is a reproduction direction speed instruction means for controlling the servo circuit 3, the reduction circuit 21, the expansion circuit 22 and the like, and a compression method or a compression ratio or a decimation method or a decimation ratio according to the reproduction speed, the reproduction direction and the intra-frame interval Change.

FIG. 5 shows the processing order and memory occupancy order of the decoder when the backward reproduction is performed in the embodiment shown in FIG.

The operation procedure of the decoder in this embodiment is shown in FIG.
It is similar to the example described in. However, in FIG. 2, four screens of the reference image memory are prepared, whereas in this embodiment,
The reference image memory is prepared only for two screens, each memory is divided into two and used, and the reference image reduced to halve the data can be stored for four screens. According to the present embodiment, similar to the conventional reproducing apparatus, it is possible to realize smooth backward reproduction only by preparing two screens of reference image memory and one screen of output memory. The compression rate can be arbitrarily set according to the I frame period and the required image quality, and thus can be applied to a signal having an arbitrary I frame period. Further, in this embodiment, the capacity of the memory group 16 is further increased,
It is also possible to perform reverse reproduction with higher image quality. Further, in the present embodiment, an example in which data is compressed by using reduction has been described, but other compression methods such as thinning, conversion, and filtering may be used.

In the above embodiments, an example in which the reference image memory and the output memory are provided separately has been described, but the two are not separated and the same memory is used as the reference image memory or the output memory is controlled by the address control of the memory. It is also possible to use Also, in the embodiments of FIGS. 4 and 5, the output memory may be divided and a part thereof may be used as a reference image memory.

In the above example, the present invention has been described as an example in which the present invention is realized by using hardware, but it is also possible to realize the present invention by using software processing.

[0037]

As described above, according to the present invention, encoding is performed using one or more prediction methods of the forward prediction method, the backward prediction method, the bidirectional prediction method, or a combination of these prediction methods. The encoded moving image data is reproduced from the recording medium, and at least a part of the image data of the frame to be referenced is decoded at the reproduction speed or the reproduction direction so as to be decoded before the data of the referenced image. Since it has a means for rearranging the data according to the above and a means for storing at least a part of the decoding result of the image data to be the frame to be referred to, even a moving image compressed using the inter-frame prediction method, With a relatively small memory capacity, smooth reverse playback can be realized.

In particular, by using thinning together, it is possible to realize smooth reverse reproduction with the same memory capacity as normal speed reproduction. Further, even in the case of a reproduction signal having an I frame period longer than the expected I frame period, smooth reverse reproduction can be performed.

[Brief description of drawings]

FIG. 1 is a block diagram showing the configuration of an embodiment of the present invention.

FIG. 2 is a diagram showing a processing order of the decoder and a memory occupying order in the case of performing −1 × speed reproduction in the embodiment of FIG. 1;

FIG. 3 is a diagram showing a processing order of the decoder and a memory occupying order in the case of performing the minus 2 × speed reproduction in the embodiment of FIG.

FIG. 4 is a block diagram showing the configuration of another embodiment of the present invention.

FIG. 5 is a diagram showing a processing order and memory occupancy order of a decoder when reverse reproduction is performed in the embodiment of FIG.

FIG. 6 is a diagram for explaining an example of an interframe prediction method.

FIG. 7 is a diagram showing a processing order of a decoder and a memory occupation order.

[Explanation of symbols]

 1 Recording Medium 2 Playback Head 3 Servo Circuit 11 Buffer 12 Variable Length Decoder 13 Data Separation Circuit 14 Inverse Quantization, IDCT Circuit 15 Adder 16 Memory Group 17 Output Terminal 18 Motion Compensation Circuit 21 Reduction Circuit 22 Enlargement Circuit 23 Playback Direction Speed instruction means

Continuation of the front page (51) Int.Cl. ⁶ Identification number Reference number within the agency FI Technical indication location H04N 7/137 Z (72) Inventor Toshifumi Sakaguchi 4-36-19 Yoyogi, Shibuya-ku, Tokyo Graphics Inc. In Communication Laboratories (72) Inventor Yusumi Wataya 4-36-19 Yoyogi, Shibuya-ku, Tokyo Within Graphics Communication Laboratories, Inc.

Claims (8)

[Claims] 1. An image signal processing apparatus for reproducing moving image coded data from a predetermined recording medium, comprising a forward prediction method,
The moving picture coded data coded using one or more prediction methods of the backward prediction method and the bidirectional prediction method, or a prediction method combining these methods is reproduced from the recording medium, and becomes a frame to be referred to. A means for rearranging the data according to the reproduction speed or the reproduction direction so that at least a part of the image data is decoded before the reference image data, and the decoding result of the image data to be the referenced frame. And a means for storing at least a part of the image signal processing apparatus. 2. A means for rearranging at least a part of image data not having a reference relationship with each other according to a reproduction speed or a reproduction direction so that the data is decoded in an output order. 1. The image signal processing device according to 1. 3. The image signal processing apparatus according to claim 1, further comprising means for compressing or decimation within a screen, a decoding result of screen data which is a frame to be referred to. 4. An in-screen compression or in-screen decimating means is a means for changing the compression method or compression ratio or decimating method or decimating ratio in accordance with a reproduction speed, a reproduction direction or an intra-frame interval. The image signal processing device according to claim 3. 5. The image signal processing apparatus according to claim 1, further comprising means for thinning out screen data which is not referred to by another image according to a reproduction speed or a reproduction direction. 6. The image signal processing apparatus according to claim 1, wherein the means for rearranging the data is realized by a reproduction servo control method. 7. The data rearranging means is realized by a storage means.
The image signal processing device according to 3, 4, or 5. 8. The image signal processing apparatus according to claim 1, wherein the means for rearranging the data is realized by software processing.