JP3548167B2 - Image processing device - Google Patents

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to an image processing method, an image processing apparatus that can use the image processing method, and a television receiver. The present invention relates to a technique for processing data encoded in accordance with, for example, the Moving Picture Expert Group (MPEG) standard.
[0002]
[Prior art]
The information handled by multimedia is enormous and diverse, and it is necessary to process such information at high speed in order to put multimedia into practical use. In order to process information at high speed, data compression / decompression technology is indispensable. As such a data compression / decompression technique, an “MPEG” method can be cited. This MPEG system is being standardized by the MPEG Committee (ISO / IEC JTC1 / SC29 / WG11) under the International Organization for Standardization (ISO) / International Electro-technical Commission (IEC). An image processing apparatus using the MPEG system is incorporated in various image-related devices such as a movie camera, a still camera, a television, a video CD playback device, and a DVD playback device.
[0003]
The video data handled by MPEG is related to a moving image, and the moving image is composed of a plurality of frames per second, for example, 30 frames, that is, still images or frames. As shown in FIG. 1, the video data has a hierarchical structure of six layers in the order of a sequence (Sequence), a GOP (Group Of Pictures), a picture (Picture), a slice (Slice), a macro block (Macroblock), and a block (Block). Consists of The number of slices constituting one picture is not constant, and the number of macroblocks constituting one slice is not constant. In FIG. 1, the macro block layer and the block layer are omitted.
[0004]
In addition, there are mainly two types of MPEG, MPEG-1 and MPEG-2, mainly due to differences in coding rates. In MPEG-1, a frame corresponds to a picture. In MPEG-2, frames or fields can correspond to pictures. Two fields constitute one frame. The structure in which a frame corresponds to a picture is called a frame structure, and the structure in which a field corresponds to a picture is called a field structure.
[0005]
MPEG uses a compression technique called inter-frame prediction. Inter-frame prediction compresses data between frames based on temporal correlation. In the inter-frame prediction, bidirectional prediction is performed. The bidirectional prediction is to use both forward prediction for predicting a current reproduced image from a past reproduced image or picture and backward prediction for predicting a current reproduced image from a future reproduced image.
[0006]
This bidirectional prediction defines three types of pictures called I-pictures (Intra-Picture), P-pictures (Predictive-Picture), and B-pictures (Bidirectionally predictive-Picture). An I picture is an image generated independently by an intra-frame encoding process independently of past and future reproduced images. In order to perform random access, at least one I picture is required in a GOP. All macroblock types in an I picture are intra-frame predictions (Intra Frame). The P picture is generated by the forward prediction, that is, the prediction from the past I picture or P picture by the inter-frame encoding process. The macroblock type in the P picture includes both an intra-frame prediction screen and a forward prediction screen (Forward Inter Frame).
[0007]
The B picture is generated by bidirectional prediction by an inter-frame encoding process. In bidirectional prediction, a B picture is generated by any one of the following three predictions.
(1) Forward prediction; prediction from past I or P pictures; (2) backward prediction; prediction from future I or P pictures; (3) bidirectional prediction; past and future I or P pictures. The macroblock types in the predicted B-picture include four types: an intra-frame prediction screen, a forward prediction screen, a backward prediction screen (Backward Inter Frame), and an interpolation prediction screen (Interpolative Inter Frame).
[0008]
These I, P, and B pictures are respectively encoded. That is, the I picture is generated without any past or future picture. In contrast, a P picture is not generated without a past picture, and a B picture is not generated without a past or future picture. However, even in the case of a P picture or a B picture, when the macroblock type is an interpolative prediction screen, the macroblock is generated even if there is no past or future picture.
[0009]
In the inter-frame prediction, first, an I picture is periodically generated. Next, a frame several frames ahead of the I picture is generated as a P picture. This P picture is generated by one-way (forward) prediction from the past to the present. Subsequently, a frame located before the I picture and after the P picture is generated as a B picture. When generating this B picture, an optimal prediction method is selected from three of forward prediction, backward prediction, and bidirectional prediction. Generally, in a continuous moving image, the current image and the images before and after the current image are very similar, and only a part of the image is different. Therefore, it is assumed that the previous frame and the next frame are the same, and if there is a change between both frames, only the difference is extracted and compressed. For example, the previous frame is an I picture, the next frame is a P picture, and the difference is extracted as B picture data. Thereby, data between frames can be compressed based on temporal correlation. A data sequence or bit stream of video data encoded according to the MPEG video part is called an MPEG video stream.
[0010]
MPEG-1 mainly corresponds to storage media such as a video CD (Compact Disc) and a CD-ROM (CD-Read Only Memory). MPEG-2 is not only a storage medium such as a video CD, a CD-ROM, a DVD (Digital Video Disk) and a VTR (Video Tape Recorder), but also a communication medium such as a LAN (Local Area Network), a terrestrial broadcast, and a satellite broadcast. And all transmission media including broadcast media such as CATV (Community Antenna Television).
[0011]
The core of the technology used in the MPEG video part is Motion Compensated prediction (MC) and Discrete Cosine Transform (DCT). An encoding technique using both MC and DCT is called a hybrid encoding technique. In the MPEG video part, a video signal of an image is decomposed into frequency components and processed using DCT (also called FDCT; Forward DCT) at the time of encoding. Then, at the time of decoding, an inverse DCT transform (inverse discrete cosine transform; IDCT; Inverse DCT) is used to return the frequency component to the video signal of the image again.
[0012]
MPEG can process an enormous amount of information at high speed. However, as described above, since a compression technique called inter-frame prediction is used, it is encoded and recorded in time series in accordance with the MPEG video part. When a data sequence is reproduced in reverse order for picture search, that is, in the reverse direction, it is very difficult to simply reproduce the recorded data sequence by going back in time as in a normal video tape recorder. Therefore, conventionally, only the I picture allocated in each GOP is reproduced retroactively on the time axis. As described above, an I picture is an image that has been subjected to intra-frame encoding processing, and therefore can be displayed independently without referring to previous and subsequent pictures.
[0013]
[Problems to be solved by the invention]
In the conventional example, the number of I-pictures allocated in each GOP is extremely small. For example, in MPEG, the number of I-pictures allocated in a GOP is at most one out of 15 to 30 pictures. When pictures of up to every 30 frames are reproduced in the reverse order, a smooth reverse order reproduction screen like a normal video tape recorder cannot be obtained, and it is difficult to stop at a desired timing with good timing. The present invention has been made in view of this point, and one of its objects is to provide an image processing technique capable of obtaining a smooth reverse-order reproduction screen.
[0014]
The present invention solves this and other objects that will become apparent from the present specification, mainly in technology related to image encoding and decoding processing.
[0015]
[Means for Solving the Problems]
One embodiment of the present invention relates to an image processing device. The apparatus includes: a pre-decoder configured to decode a first encoded data sequence including an I picture, a P picture, and a B picture encoded according to MPEG to generate a first reproduced image data sequence; A first frame memory that is used for decoding by the predecoder and stores a first reproduced image data sequence, and a first reproduced image data sequence read from the first frame memory are An encoder for re-encoding to generate a second encoded data sequence, a post-decoder for decoding the second encoded data sequence to generate a second reproduced image data sequence, A second frame memory used for decoding by the decoder and storing at least one picture included in the second reproduced image data sequence.
[0016]
The first frame memory stores a first area storing at least one I picture or P picture that is forward-referenced when decoding a P picture or a B picture, and an I picture or P picture that is backward-referenced when decoding a B picture. It has a second area for storing at least one, and a third area for storing at least one B picture, which is used for decoding by the pre-decoder. In other words, the frame memory in this embodiment requires a capacity of at least four frames in total, including the first frame memory and the second frame memory.
[0017]
Another embodiment of the present invention also relates to an image processing apparatus. The apparatus includes: a pre-decoder configured to decode a first encoded data sequence including an I picture, a P picture, and a B picture encoded according to MPEG to generate a first reproduced image data sequence; A frame memory that is used for decoding by the pre-decoder and stores the first reproduced image data sequence, and re-encodes the first reproduced image data sequence read from the frame memory to generate a second reproduced image data sequence And a post-decoder that decodes the second encoded data sequence to generate a second reproduced image data sequence.
[0018]
The frame memory stores a first area for storing at least one I picture or P picture that is forward referenced in decoding a P picture or a B picture, and at least one I picture or P picture that is backward referenced for decoding a B picture. And a third area used for decoding by the pre-decoder and storing at least one B picture. The third area is used for decoding by the post-decoder, and is also used as an area for storing a picture decoded by the post-decoder. The frame memory in this embodiment requires a capacity of at least three frames.
[0019]
Still another preferred embodiment according to the present invention relates also to an image processing apparatus. The apparatus includes: a pre-decoder configured to decode a first encoded data sequence including an I picture, a P picture, and a B picture encoded according to MPEG to generate a first reproduced image data sequence; A frame memory for storing the first reproduced image data sequence and a first reproduced image data sequence read from the predecoder are re-encoded while being used for decoding by the predecoder. An encoder for generating a second encoded data sequence and a post-decoder for decoding the second encoded data sequence to generate a second reproduced image data sequence are provided.
[0020]
The frame memory stores a first area for storing at least one I picture or P picture that is forward referenced in decoding a P picture or a B picture, and at least one I picture or P picture that is backward referenced for decoding a B picture. And a third area that is used for decoding by the pre-decoder and that can store at least one picture. The third area is used for decoding by the post-decoder, and is also used as an area for storing a picture decoded by the post-decoder. The frame memory in this embodiment requires a capacity of at least three frames.
[0021]
Still another preferred embodiment according to the present invention relates also to an image processing apparatus. This apparatus includes a plurality of decoders for decoding an encoded data sequence for different purposes, a frame memory for storing a reproduced image data sequence generated by decoding of these decoders, and a reproduced image data sequence for converting a video signal into a video signal. And a display circuit for conversion.
[0022]
In the frame memory, when at least a part of the data of the reproduced image data string generated by decoding by any of the decoders is output to the display circuit, the output data is generated by decoding by another decoder. By overwriting with the reproduced image data sequence, a partial area included in the frame memory is shared by a plurality of decoders. The plurality of decoders include a first decoder that decodes an encoded data sequence read in time series and a second decoder that decodes an encoded data sequence read in anti-time sequence. And the reproduced image data sequence generated by the decoding of the first decoder is generated by the decoding of the second decoder without waiting for this to be converted into a video signal by the display circuit. It may be overwritten with a reproduced image data string.
[0024]
It should be noted that any one of the above image processing apparatuses may be mounted to provide a television receiver having anti-time-series image reproduction as a part of its operation specifications. “I picture”, “B picture”, and “P picture” correspond to “I-VOP (Video Object Plane)”, “B-VOP”, and “P-VOP” in MPEG4, respectively. Include.
[0025]
As described above, in any case, the encoding or decoding, and possibly the accompanying processing, may be performed in a predetermined group unit. Further, different combinations of the above-described arbitrary components, processing steps, and the like, and those obtained by converting the expressions of the present invention among methods, apparatuses, systems, computer programs, recording media, and the like are also effective as embodiments of the present invention. .
[0026]
BEST MODE FOR CARRYING OUT THE INVENTION
An embodiment of the present invention will be described. Processing common to some of the embodiments is forward and reverse image playback. In the following description, the terms “normal order”, “reverse order”, and their synonyms refer to the order in which pictures constituting an image are finally displayed for convenience of explanation. For this reason, unless otherwise noted, the order of the pictures is based on the display state.
[0027]
As described later, even in the reverse reproduction, the pictures in each GOP of the MPEG data stream are first decoded in the forward direction, that is, in time series. This is re-encoded, and in the subsequent re-decoding stage, the reverse order is realized for the first time. Therefore, “reverse order” in reverse order reproduction mainly relates to the second decoding. Various combinations of the order of the I, P, and B pictures can be considered in the MPEG bit stream. As a frame memory included in the configuration of each embodiment, for example, an SDRAM (Synchronous Dynamic RAM), a DRAM, a Rambus DRAM, or the like is used.
[0028]
In the following embodiments, various constituent members appear. These can be realized by hardware, such as a CPU, a memory, and other LSIs and combinational circuits, and can be realized by software, such as a program having an image processing function loaded into a memory. Focusing on the functions realized by the cooperation of Therefore, it is understood by those skilled in the art that these functions can be realized in various forms by hardware only, software only, or a combination thereof. The image reproducing device is an example of the “image processing device” of the present invention.
[0029]
(1st Embodiment)
The present embodiment exemplifies a configuration that can be considered when each block constituting the image processing apparatus is tentatively formed into one chip. The configuration of the present embodiment does not particularly consider the correspondence with the present invention, and is merely an example as a target to be compared with the second embodiment and thereafter.
FIG. 2 shows a block circuit of the image reproducing apparatus 1 according to the first embodiment. The image playback device 1 is incorporated in a movie camera, a still camera, a television, a video CD playback device, a DVD playback device, or the like that outputs an MPEG video stream from the transmission medium 2 to a display 3. The transmission medium 2 includes storage media (video CD, CD-ROM, DVD, VTR, etc.), communication media (LAN, etc.), broadcast media (terrestrial broadcasting, satellite broadcasting, CATV, etc.). When the data from the storage medium or the broadcast medium is data that has not been encoded according to the MPEG video part, the transmission medium also includes an MPEG video encoder for encoding the digital data. When the image reproducing apparatus 1 is incorporated in a movie camera or a still camera, the transmission medium 2 is replaced with an imaging device such as a CCD and its signal processing circuit.
[0030]
2, the image reproducing apparatus 1 includes a hard disk (HD) 4, an MPEG video decoder 5 (hereinafter, also simply referred to as “decoder 5”), an MPEG video encoder 6 (hereinafter, simply referred to as “encoder 6”), and a second. MPEG video decoder 7 (hereinafter also simply referred to as "second decoder 7"), first frame memory 52, second frame memory 62, third frame memory 72, first display circuit 54, video input circuit 64, second It comprises a display circuit 74, a switching circuit 8, and a control core circuit 10. The control core circuit 10 controls the operations of the decoder 5, the second decoder 7, the encoder 6, and the components of the image reproducing apparatus 1. The hard disk 4 is composed of a magnetic disk, and sequentially accumulates video streams transferred from the transmission medium 2. A special storage area 4a is provided in the hard disk 4.
[0031]
The decoder 5, the first frame memory 52, and the first display circuit 54 constitute a first block 50, and the encoder 6, the second frame memory 62, and the video input circuit 64 constitute a second block 60. The second decoder 7, the third frame memory 72, and the second display circuit 74 constitute a third block 70. The entire image reproducing apparatus 1 or its main part may be mounted on a one-chip LSI, and the same applies to other embodiments. In the case of the present embodiment, each of the first block 50, the second block 60, and the third block 70 is mounted on a one-chip LSI. However, each of the first frame memory 52, the second frame memory 62, and the third frame memory 72 may be mounted externally.
[0032]
The reproduced image data sequence generated by the decoder 5 is temporarily stored in the first frame memory 52, and is sequentially input to the first display circuit 54. The first frame memory 52 has a function as a display buffer and a function as a work area for the decoder 5. The first display circuit 54 generates a video video signal from the picture data transferred from the decoder 5 and outputs the video video signal to the display 3 connected to the image reproducing device 1 via the switching circuit 8.
[0033]
The video video signal output from the first block 50 is also input to the video input circuit 64. The video input circuit 64 converts the video video signal into a reproduced image data sequence, and the reproduced image data sequence is temporarily stored in the second frame memory 62, and then sequentially re-encoded by the encoder 6. The second frame memory 62 also has a function as a work area of the encoder 6.
[0034]
After the coded data sequence output from the second block 60 is stored in the storage area 4a, it is read out in a time-series manner and the second decoder 7 re-decodes it. The re-decoded reproduced image data string is temporarily stored in the third frame memory 72 and then converted by the second display circuit 74 into a video signal. The third frame memory 72 has a function as a display buffer and a function as a work area for the second decoder 7. This video signal is output to the display 3 connected to the image reproducing device 1 via the switching circuit 8.
[0035]
The switching circuit 8 switches the connection to the first node 8a and the second node 8b under the control of the control core circuit 10. When the switching circuit 8 is connected to the first node 8a, normal reproduction based on the output of the first block 50 is performed. When connected to the second node 8b, reverse reproduction is performed based on the output of the third block 70.
[0036]
The first frame memory 52 is divided into a first area 56, a second area 57, and a third area 58. The first area 56 has a capacity to store at least one I picture or P picture that is referred to forward when decoding a P picture or a B picture. The second area 57 has a capacity to store at least one I picture or P picture that is referred to backward when decoding a B picture. The third area 58 has a capacity to store at least one B picture. This third area is also used as a work area used for decoding processing by the decoder 5. The second frame memory 62 has a capacity to store at least one picture included in the reproduced image data sequence input by the video input circuit 64. The third frame memory 72 has a capacity to store at least one picture included in the reproduced image data sequence decoded by the second decoder 7. The first frame memory 52, the second frame memory 62, and the third frame memory 72 together have a capacity of at least five frames.
[0037]
FIG. 3 is a block diagram showing a configuration of the decoder 5. Referring to FIG. 1, the decoder 5 includes a Huffman decoding circuit 14, an inverse quantization circuit 15, an IDCT (Inverse Discrete Cosine Transform) circuit 16, an MC (Motion Compensated prediction) circuit 17, and ROMs (Read Only Memory) 18, 19. I have. Note that the decoder 5 is an example of the “predecoder” of the present invention in another embodiment.
[0038]
The Huffman decoding circuit 14 performs variable length decoding on the picture read from the hard disk 4 based on the Huffman code stored in the Huffman table stored in the ROM 18. The inverse quantization circuit 15 performs an inverse quantization on the decoding result of the Huffman decoding circuit 14 based on a quantization threshold stored in a quantization table stored in the ROM 19 to obtain a DCT (Discrete Cosine Transform) coefficient. Ask. The IDCT circuit 16 performs IDCT on the DCT coefficient obtained by the inverse quantization circuit 15. The MC circuit 17 performs an MC on the processing result of the IDCT circuit 16.
[0039]
In this manner, the decoder 5 decodes the input MPEG video stream and generates a time-series continuous reproduced image data sequence. Note that this MPEG video stream is an example of the “first encoded data sequence” of the present invention in another embodiment.
[0040]
FIG. 4 is a block diagram illustrating a configuration of the encoder 6. 2, the encoder 6 includes an MC circuit 20, a DCT circuit 21, a quantization circuit 22, a Huffman encoding circuit 23, and ROMs 24 and 25. Note that the encoder 6 is an example of the “encoder” of the present invention in another embodiment.
[0041]
The DCT circuit 21 captures the reproduced image data input from the decoder 5 in block units, performs two-dimensional discrete cosine transform, and generates DCT coefficients. The quantization circuit 22 quantizes the DCT coefficient with reference to a quantization threshold stored in a quantization table stored in the ROM 24. Note that the ROM 24 may also be used as the ROM 19.
[0042]
The Huffman encoding circuit 23 performs variable length encoding of the quantized DCT coefficients with reference to the Huffman code stored in the Huffman table stored in the ROM 25, and thereby compresses the compressed image data in units of a screen. Generate. Note that the ROM 25 may also be used as the ROM 18.
[0043]
In this manner, the encoder 6 re-encodes the time-series continuous reproduced image data sequence to generate an MPEG video stream. This MPEG video stream is an example of the “second encoded data sequence” of the present invention in another embodiment.
[0044]
FIG. 5 is a block diagram showing a configuration of the second decoder 7. 2, the second decoder 7 includes a Huffman decoding circuit 26, an inverse quantization circuit 27, an IDCT circuit 28, an MC circuit 29, and ROMs 30 and 31. Note that the second decoder 7 is an example of the “post-decoder” of the present invention in another embodiment.
[0045]
The configuration of the second decoder 7 is similar to the configuration of the decoder 5. Therefore, the Huffman decoding circuit 26 has the same circuit configuration as the Huffman decoding circuit 14, the inverse quantization circuit 27 has the inverse quantization circuit 15, the IDCT circuit 28 has the IDCT circuit 16, and the MC circuit 29 has the same circuit configuration as the MC circuit 17. Have. The ROM 30 may also serve as the other ROM 18 or ROM 25, and the ROM 31 may serve as the ROM 19 or ROM 24.
[0046]
Based on the above configuration, the operation of reverse reproduction in the image reproducing device 1 of the present embodiment will be described with reference to the flowchart shown in FIG. The operation of the image reproducing device 1 is executed under the control of the control core circuit 10. Here, it is assumed that the MPEG video stream is composed of i GOPs (GOP _{0 to} GOP _i−1 ).
[0047]
In the reverse order reproduction, each GOP is sequentially processed from GOP _i-1 retroactively on the time axis. However, the pictures in each GOP are decoded in the decoder 5 in the forward direction, that is, in time series. When the reverse reproduction is instructed, the switching circuit 8 is connected to the second node 8b (S1), and the MPEG video stream corresponding to GOP _i-1 is read out from the hard disk 4 in picture units and input to the decoder 5, Reproduced image data for each screen is sequentially generated in time series and input to the encoder 6 (S2). The encoder 6 re-encodes the reproduced image data sequence for one GOP input from the decoder 5 into an I picture or the like (S3). The re-encoded data sequence for one GOP from the encoder 6 is overwritten on the storage area 4a of the hard disk 4 (S4).
[0048]
When the writing to the storage area 4a is completed, the re-encoded data sequence stored in the storage area 4a is read out in an anti-time sequence, that is, retroactively on the time axis, and the second decoder 7 sequentially decodes the read out data sequence. Output to the second display circuit 74 (S5). When the writing to the storage area 4a is completed, a write end signal is sent out, an MPEG video stream corresponding to the next GOP _i-2 is input to the decoder 5, and the processing from S2 is performed. That is, in S5, when the data sequence of one GOP is being decoded by the second decoder 7, the decoder 5 is decoding the data sequence of the next one GOP. Reproduced image data is input to the second display circuit 74 from the second decoder 7 in a reverse chronological order, and a reverse reproduction screen is displayed on the display 3.
[0049]
Next, the operation for normal reproduction will be described with reference to the flowchart shown in FIG. In normal order reproduction, processing is performed sequentially from GOP ₀ according to the time axis. The pictures in each GOP are naturally decoded in the decoder 5 in the forward direction. When forward reproduction is instructed, the switching circuit 8 is connected to the first node 8a (S11), and the MPEG video stream corresponding to GOP ₀ is read out from the hard disk 4 in picture units and input to the decoder 5 for reproduction. Image data is sequentially generated in chronological order on a screen basis, and is input to the encoder 6 and the first display circuit 54 in parallel (S12). The first display circuit 54 generates a video signal based on the input playback image data in screen units and outputs the video signal to the display 3, whereby a normal playback screen is displayed on the display 3 (S14).
[0050]
On the other hand, in parallel with the processing of the first display circuit 54, the encoder 6 re-encodes the reproduced image data sequence for one GOP input from the decoder 5 into an I picture or the like (S15). The re-encoded data string is overwritten on the storage area 4a of the hard disk 4 (S16). When the processing of GOP ₀ is completed, the process returns to S12 and the processing of the next GOP ₁ is performed. That is, during normal-order reproduction, the encoder 6 sequentially re-encodes the same image data sequence into I pictures or the like in GOP units.
[0051]
(2nd Embodiment)
Embodiments after this embodiment have a configuration corresponding to the present invention. The second embodiment is a more compact design of the first embodiment. The second embodiment differs from the image reproducing apparatus 1 of the first embodiment in that the frame memory is configured with a smaller capacity as a whole. That is, the first frame memory 52 and the second frame memory 62 in the first embodiment are shared, and are configured with a capacity of four frames as a whole.
[0052]
FIG. 8 shows a block circuit of an image reproducing apparatus 1 according to the second embodiment. In the first embodiment, the video video signal read from the first display circuit 54 is converted into a reproduced image data sequence and then re-encoded, but in the present embodiment, the video signal is reproduced by the first display circuit 54. A reproduced image data sequence before being converted into a video signal is read out and re-encoded. Therefore, the configuration corresponding to the second frame memory 62 and the video input circuit 64 in the first embodiment is not included in the present embodiment. The first frame memory 80 is divided into a first area 82, a second area 84, and a third area 86, and correspond to the first area 56, the second area 57, and the third area 58 of the first embodiment, respectively. However, the third area 86 is different from the third area 58 of the first embodiment in that the third area 86 is also used as a work area for the encoding process by the encoder 6. The second frame memory 90 corresponds to the third frame memory 72 of the first embodiment.
[0053]
Since the reproduced image data sequence generated by the decoder 5 during the reverse reproduction is not displayed on the display 3 as it is, it is sufficient to input it to the encoder 6, and it is not actually necessary to input it to the first display circuit 54. . Therefore, after the data string is transferred from the third area 86 to the encoder 6, the encoder 6 sets the third area 86 as a work area without waiting for conversion to a video signal by the first display circuit 54. You can safely start using it.
[0054]
The encoded data sequence re-encoded by the encoder 6 is stored in the storage area 4a in chronological order. This is read out in an anti-chronological order and re-decoded by the second decoder 7, thereby realizing reverse-order reproduction.
[0055]
The internal configurations of the decoder 5, the encoder 6, and the second decoder 7 in the present embodiment may be the same as those in FIGS. 3, 4, and 5, respectively, and the overall operation flow may be the same as in FIGS.
[0056]
As described above, the image reproducing apparatus 1 according to the present embodiment has the following operational effects.
(1) A frame memory that is used as a display buffer and a work area for encoding and decoding only needs to have a capacity of four frames, so that the memory capacity can be reduced as compared with the configuration of the first embodiment, and cost can be reduced. Also leads to.
(2) Since the second frame memory 62 and the video input circuit 64 are unnecessary compared to the first embodiment, the size and cost are reduced.
(3) The second decoder 7 decodes the coded data sequence read out in a time-series manner, whereby reverse reproduction can be realized.
[0057]
(Third embodiment)
The third embodiment is a further compact design of the second embodiment. The third embodiment differs from the image reproducing apparatus 1 of the second embodiment in that a frame memory is configured with a smaller capacity as a whole. In other words, the first frame memory 80 and the second frame memory 90 in the second embodiment are shared, and are configured with a capacity of three frames as a whole.
[0058]
FIG. 9 shows a block circuit of an image reproducing apparatus 1 according to the third embodiment. The frame memory 100 is divided into a first area 102, a second area 104, and a third area 106, and correspond to the first area 82, the second area 84, and the third area 86 of the second embodiment, respectively. However, the third region 106 is different from the third region 86 of the second embodiment in that the third region 106 is also used as a work area for decoding processing by the second decoder 7. The display circuit 110 corresponds to the first display circuit 54 of the second embodiment. Configurations corresponding to the second frame memory 90, the second display circuit 74, and the switching circuit 8 in the second embodiment are not included in the present embodiment.
[0059]
At the time of reverse reproduction, the timing at which the picture stored in the third area 106 is output to the display circuit 110 and the timing at which this picture is overwritten with the picture generated by the decoder 5 are adjusted to adjust the third area. 106 is realized. For example, the line currently displayed on the display 3 may be grasped, and the timing may be adjusted so as to overwrite data corresponding to the displayed line. This may be adjusted on a macroblock basis.
[0060]
On the other hand, since the reproduced image data sequence generated by the decoder 5 during the reverse reproduction is not displayed on the display 3 as it is, after the data sequence is transferred from the third area 106 to the encoder 6, the display circuit 110 The timing may be adjusted so as to start overwriting with the reproduced image data sequence generated by the second decoder 7 without waiting for conversion to the video video signal.
[0061]
The internal configurations of the decoder 5, the encoder 6, and the second decoder 7 in the present embodiment may be the same as those in FIGS. 3, 4, and 5, respectively, and the overall operation flow may be the same as in FIGS.
[0062]
As described above, the image reproducing apparatus 1 according to the present embodiment has the following operational effects.
(4) The capacity of the frame memory used as the display buffer and the work area for encoding and decoding is sufficient for three frames, so that the memory capacity can be further reduced, and the cost can be further reduced.
(5) Compared to the first embodiment, the second frame memory 62, the video input circuit 64, the third frame memory 72, the second display circuit 74, and the switching circuit 8 are unnecessary, so that further reduction in size and cost can be achieved. Connect.
[0063]
(Fourth embodiment)
The fourth embodiment has a configuration that is a modification of the third embodiment. The fourth embodiment is different from the image reproducing apparatus 1 of the third embodiment in that the reproduced image data sequence read directly from the decoder 5 is encoded instead of the reproduced image data sequence read from the frame memory 100. On the point. The frame memory 100 has a capacity of three frames, like the frame memory 100 of the third embodiment.
[0064]
FIG. 10 shows a block circuit of an image reproducing apparatus 1 according to the fourth embodiment. The reproduced image data string output from the decoder 5 is input not only to the frame memory 100 but also to the encoder 6. The third area 106 is also used as a work area for the encoding process of the encoder 6.
[0065]
In the present embodiment, similarly to the third embodiment, the reproduced image data sequence generated by the decoder 5 at the time of reverse reproduction is converted into a video signal by the display circuit 110 without waiting for conversion to a second video signal. 7 may be adjusted so as to overwrite with the reproduced image data sequence generated by step 7. However, since the reproduced image data string stored in the third area 106 is not transferred to the encoder 6, there is no need to adjust the timing with the timing.
[0066]
The internal configurations of the decoder 5, the encoder 6, and the second decoder 7 in the present embodiment may be the same as those in FIGS. 3, 4, and 5, respectively, and the overall operation flow may be the same as in FIGS.
[0067]
As described above, the image reproducing apparatus 1 according to the present embodiment has the following operational effects.
(6) As in the third embodiment, the frame memory used as the display buffer and the work area for encoding and decoding only requires a capacity of three frames, and the memory capacity can be further reduced, contributing to further cost reduction. can do.
(7) Similar to the third embodiment, the second frame memory 62, the video input circuit 64, the third frame memory 72, the second display circuit 74, and the switching circuit 8 are unnecessary as compared with the first embodiment. This leads to downsizing and cost reduction.
(8) Since the encoding process by the encoder 6 is started without waiting for the reproduction image data sequence generated by the decoder 5 to be written to the frame memory 100, as long as the encoding process of the encoder 6 maintains a sufficient processing speed. As a result, the overall processing speed is increased.
[0068]
(General Consideration on Embodiment)
As will be appreciated by those skilled in the art, any combination of the embodiments not described above is possible. For example, the following considerations or modifications are possible.
[0069]
(A) As the hard disk 4, a magneto-optical disk, an optical disk, or the like is used instead of a magnetic disk.
(B) As the hard disk 4, a rewritable semiconductor memory, for example, an SDRAM, a DRAM, a Rambus DRAM or the like is used.
(C) The hard disk 4 and the storage area 4a are provided independently. In this case, a rewritable semiconductor memory is desirable for the storage area 4a.
[0070]
(D) Extract a data string from the MPEG video stream in the following units instead of one GOP. The following units including the GOP are also included in the concept of the group unit.
-A unit starting from a P picture is set as a GOP without using a unit starting from an I picture as a GOP.
-Several pictures are grouped without being bound by the concept of GOP.
-Change the number of pictures arbitrarily for each group.
[0071]
(E) A RAM (Random Access Memory) is used instead of the ROMs 18, 19, 24, 25, 30, 31.
(F) An operation key for selecting the reverse order reproduction function is provided on the image reproduction apparatus.
(G) Play back one frame at a time in reverse order according to key operation.
[0072]
(H) In each embodiment, the pictures included in the re-encoded data sequence generated by the encoder 6 are written in the storage area 4a in chronological order. The data may be written in the storage area 4a. In this case, the second decoder 7 does not need to read out the re-encoded data sequence from the storage area 4a in a time-series manner.
(I) In the second, third and fourth embodiments, the decoder 5 and the second decoder 7 are shared in hardware. In the second embodiment, the first display circuit 54 and the second display circuit 74 are shared in hardware.
[0073]
(J) In addition to the embodiments described above, there are the following forms as applications having two encoding or decoding functions in one device. Therefore, in the second and fourth embodiments, the example in which the decoder 5 and the second decoder 7 are shared as the shared decoder 53 has been described. However, in the case where two encoders are provided, these encoders are shared. Is also good.
(I) A case where a movie camera simultaneously shoots a subject from different viewpoints, and compresses and expands the data by the MPEG method.
(Ii) In a television, a plurality of programs are simultaneously decoded and displayed on two screens.
(Iii) In television, a case where a plurality of programs are simultaneously decoded and channel switching is performed seamlessly. In broadcasting using MPEG, once decoding is interrupted due to channel switching or the like, it takes some time for 0.5 to 2 seconds until a new sequence header is detected, and then it takes a little time to restart, and usually during that time, The picture freezes or blacks out, but (iii) is effective to solve this problem.
(Iv) In a television connected to a DVD, a digital still camera, or the like, the broadcast and the DVD or the digital still camera are simultaneously reproduced.
(V) When the program or the counterprogram is recorded in a moving image or a still image state during the reproduction of the program, and the recorded moving image or the still image and the broadcast program are simultaneously superimposed and reproduced.
(Vi) A case where a reproduced image is coded by the JPEG method at predetermined time intervals and taken into a ring buffer, and is used as an index for jumping to a near scene in a reverse search.
[0074]
(K) For reverse order reproduction, image data for one GOP had to be completely stored in the storage area 4a. This is because data is read out only in the forward direction inside the GOP, and a picture cannot be generated during reverse playback unless all data for one GOP is left. For this reason, a storage capacity for recording image data for one GOP is determined in the storage area 4a. On the contrary, in the first embodiment and the like, the encoder 6 is free-running even during the normal reproduction, and the data for the reverse reproduction of one GOP is always generated and stored in the first embodiment. Thereby, the switching from the normal order to the reverse order is intended to be smooth.
[0075]
However, in this method, although the reproduction direction can be changed more smoothly than when free-run is not performed, a time lag does not necessarily occur at the time of switching. This is because, when the reverse reproduction is performed on GOP _n, the _{GOP n-1} of the previous, it is necessary to the decoder 5 decodes reads the encoded data of 1GOP fraction, a series of processes GOP _n This is because it may not be completed by the completion of the reverse reproduction. If it does not end, reverse playback will stop there for a moment.
[0076]
To cope with this problem, the storage of image data for one GOP described in the first embodiment and the like is extended to store image data for a maximum of about two GOPs. It can be completely eliminated. Therefore, if such specifications are required, this measure may be taken.
[0077]
When the number of pixels is reduced, decoding in a down-conversion format may be performed in advance in the IDCT processing in the decoder 7. That is, in a normal case, for example, a place where IDCT is to be performed on a square block of 8 × 8 pixels may be performed on a half-size block of 8 × 4 pixels. In this case, since the capacity of the image data to be stored in the frame memory at the time of image reproduction is reduced to １ ／, as a result, the picture of 2 GOPs can be stored in the empty area. When this down-conversion is performed, the image of 1960 × 1080 pixels in the high definition mode becomes 980 × 1080 pixels. Therefore, at the time of reproduction, resolution reproduction processing is performed, such as displaying each pixel twice in the horizontal direction.
[0078]
(L) The above-described time lag at the time of switching should also be considered when shifting from reverse-order reproduction to normal-order reproduction. At this time, the same measures as described above, that is, by storing the data of the read picture for about 1 to 2 GOPs can be dealt with. Now, assuming that the n-th GOP _n is being read for reverse reproduction, the data of the picture of this GOP _n is read until the read for reverse reproduction reaches the GOP _n-2 immediately before that. Keep it. That is, by holding the data of a certain GOP until the reading of the data of the two previous GOPs, seamless reproduction is realized even when switching to normal reproduction is performed.
[0079]
Switching from reverse-order reproduction to normal-order reproduction can be dealt with only by the processing of the decoder 5, so that the time lag is originally smaller than in the case of (k). Therefore, although 2 GOPs were set here, it is considered that 1 GOP or more is actually sufficient at best. However, since this value depends on the mounting of the device, it is desirable to determine the value by experiment or the like for each model.
[0080]
【The invention's effect】
According to the present invention, it is possible to provide an image processing technique capable of performing smooth reverse reproduction.
[Brief description of the drawings]
FIG. 1 is an explanatory diagram showing a hierarchical structure of an MPEG video stream.
FIG. 2 is a block circuit diagram of the image reproducing apparatus according to the first embodiment.
FIG. 3 is a block diagram schematically illustrating a decoder according to the first embodiment.
FIG. 4 is a block diagram schematically illustrating an encoder according to the first embodiment.
FIG. 5 is a block diagram schematically illustrating a decoder according to the first embodiment.
FIG. 6 is a flowchart illustrating a reverse reproduction operation of the image reproduction device according to the first embodiment.
FIG. 7 is a flowchart illustrating a forward reproduction operation of the image reproduction device according to the first embodiment.
FIG. 8 is a block circuit diagram of an image reproducing apparatus according to a second embodiment.
FIG. 9 is a block circuit diagram of an image reproducing device according to a third embodiment.
FIG. 10 is a block circuit diagram of an image reproducing apparatus according to a fourth embodiment.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 1 Image reproducing apparatus 2 Transmission media 3 Display 4 Hard disk 4a Storage area 5, 7 MPEG video decoder 6 MPEG video encoder 8 Switching circuit 10 Control core circuit 14, 26 Huffman decoding circuit 15, 27 Inverse quantization circuit 16, 28 IDCT circuit 17 , 20, 29 MC circuit 21 DCT circuit 22 Quantization circuit 23 Huffman coding circuit 18, 19, 24, 25, 30, 31 ROM
52, 62, 72, 80, 90, 100 Frame memories 54, 74, 110 Display circuit 64 Video input circuit

Claims (4)

A pre-decoder that decodes a first encoded data sequence including an I picture, a P picture, and a B picture encoded according to MPEG to generate a first reproduced image data sequence;
A first frame memory used for decoding by the pre-decoder and storing the first reproduced image data sequence;
An encoder configured to re-encode the first reproduced image data sequence read from the first frame memory to generate a second encoded data sequence;
A post-decoder that decodes the second encoded data sequence to generate a second reproduced image data sequence;
A second frame memory that is used for decoding by the post-decoder and stores at least one picture included in the second reproduced image data sequence;
With
The first frame memory includes a first area for storing at least one I picture or P picture that is forward-referenced when decoding a P picture or a B picture, and an I picture or P picture that is backward-referenced when decoding a B picture. An image processing apparatus comprising: a second area storing at least one B picture; and a third area storing at least one B picture, which is used for decoding processing by the pre-decoder. . A pre-decoder that decodes a first encoded data sequence including an I picture, a P picture, and a B picture encoded according to MPEG to generate a first reproduced image data sequence;
A frame memory used for decoding by the pre-decoder and storing the first reproduced image data sequence;
An encoder for re-encoding the first reproduced image data sequence read from the frame memory to generate a second encoded data sequence;
A post-decoder that decodes the second encoded data sequence to generate a second reproduced image data sequence;
With
The frame memory includes a first area for storing at least one I picture or P picture that is forward referenced in decoding a P picture or a B picture, and at least one I picture or P picture that is backward referenced in decoding a B picture. A second area for storing, and a third area which is used for decoding processing by the pre-decoder and stores at least one B picture, and wherein the third area is An image processing device, which is an area used for decoding processing by the pre-decoder and is also used as an area for storing a picture decoded by the post-decoder. A pre-decoder that decodes a first encoded data sequence including an I picture, a P picture, and a B picture encoded according to MPEG to generate a first reproduced image data sequence;
A frame memory used for decoding by the pre-decoder and storing the first reproduced image data sequence;
An encoder for re-encoding the first reproduced image data sequence read from the pre-decoder to generate a second encoded data sequence;
A post-decoder that decodes the second encoded data sequence to generate a second reproduced image data sequence;
With
The frame memory includes a first area for storing at least one I picture or P picture that is forward referenced in decoding a P picture or a B picture, and at least one I picture or P picture that is backward referenced in decoding a B picture. A second area for storing, and a third area which is used for decoding processing by the pre-decoder and is capable of storing at least one picture, and wherein the third area is An image processing device, which is an area used for decoding processing by the pre-decoder and is also used as an area for storing a picture decoded by the post-decoder. A plurality of decoders for decoding the encoded data sequence for different purposes,
A frame memory for storing a reproduced image data sequence generated by decoding by these decoders,
A display circuit for converting the reproduced image data sequence into a video signal,
With
In the frame memory, when at least a part of data of a reproduced image data string generated by decoding of any of the decoders is output to the display circuit, the output data is generated by decoding of another decoder. In the image processing apparatus, by overwriting with the reproduced image data sequence, a partial area included in the frame memory is shared by a plurality of the decoders,
The plurality of decoders include a first decoder that decodes an encoded data sequence read in time series, and a second decoder that decodes an encoded data sequence read in anti-time sequence. Vessels and
The reproduced image data sequence generated by the decoding of the first decoder is a reproduced image generated by the decoding of the second decoder without waiting for this to be converted into a video signal by the display circuit. An image processing apparatus characterized by being overwritten with a data string.

Images