WO1998044745A1

WO1998044745A1 - Apparatus and method for simultaneous video decompression

Info

Publication number: WO1998044745A1
Application number: PCT/JP1998/001432
Authority: WO
Inventors: Bryan Severt Hallberg; Shaw-Min Lei; Dean Savage Messing; Jeffrey Brian Sampsell; Muhammed Ibrahim Sezan; Larry Alan Westerman
Original assignee: Sharp Kabushiki Kaisha
Priority date: 1997-03-31
Filing date: 1998-03-30
Publication date: 1998-10-08

Abstract

The apparatus is intended to simultaneously decompress separated portions of a compressed video image. The apparatus includes an input mechansim (12) for receiving a compressed video image. A video decoder mechanism (18) is provided for decompressing, asynchronously, the compressed video image. The image is partitioned into discrete slices, each slice is decoded by a discrete segment of the video decoder mechanism. A selected, or boundary, portion of each slice may be decoded by more than one segment of the video decoder mechanism. A signal director (32) is provided for directing the decoded video image to plural video image buffers (46), each having plural memory locations therein, which are provided for storing a single video image. A video output mechanism (54) is provided for transmitting a decompressed video image. The method includes partitioning the video image into discrete slices; designating a portion of each slice to be a shared portion and designating the remainder of each slice to be an exclusive portion; defining a buffer having multiple memory locations therein; storing the image in the buffer, which includes storing the exclusive portion of each slice in a single memory location and storing the shared portion of the image in at least two memory locations; decoding, asynchronously, each slice with a discrete video decoder, wherein a video decoder reads only the exclusive portion of its assigned slice and reads both the shared portion of its assigned slice and the shared portion of an adjacent slice to form a decoded slice; synchronizing the decoded slices; and writing the decoded image to a video output mechanism.

Description

DESCRIPTION APPARATUS AND METHOD FOR SIMULTANEOUS VIDEO DECOMPRESSION

Field of the Invention This invention relates to high resolution digital television, and specifically to an apparatus and method for decompressing a digital video signal.

Background of the Invention A significant advantage of digital television is that the signal comprising the video and audio information may be compressed, by any number of data compression techniques. This allows for higher speed transmission of the signal and allows more information to be contained within a specific signal. The signal, however, must be decompressed so that the image may be displayed on a television screen and the audio portion may be played through a speaker. The compression and decompression of audio and video signals is a computer intensive operation, requiring a great deal of information to be read, decompressed and assembled into the video and audio components of the digital television signal. Of particular interest is the decompression of signals generated under the Motion

Picture Experts Group (MPEG) standards, or any other decompression or image processing system wherein the system reaches both upwards and downwards to sequentially arranged images. One way to decompress such a signal is to partition the signal into "slices", store the slices in memory, and provide multiple decoder mechanism, such as decoder chips, which work in an asynchronous, parallel manner to decode their particular slice. In the case where the portion of the image near the slice boundary is being decoded, more than one chip will require access to the memory location for that slice portion, in order to avoid artifacts along the line that is being decoded.

Known memory access bus management hardware and software, if given the aforementioned task, will operate only as fast as the slowest decoding chip in the system, causing the other decoder chips to pause, or wait, until memory access by the slowest decoder chip has been completed. Additionally, the use of a shared bus by plural chips limits the rate of data flow.

This obviously causes delays in the decoding process and limits the amount of information which may be decoded for a given time period. U.S. Patent Nos. 5,428,403, 5,473,379 and 5,475,430 disclose techniques wherein a video frame is divided into multiple blocks for compression and transmission, and which use motion vectors to decompress and align the multiple blocks into a final image.

Summary of the Invention The apparatus of the invention is intended to simultaneously decompress separated portions of a compressed video image. The apparatus includes an input mechanism for receiving a compressed video image. Additionally, a no-wait video decoder mechanism is provided for decompressing, asynchronously, the compressed video image. The image is partitioned into discrete slices, each slice is decoded by a discrete segment of the video decoder mechanism. A selected, or boundary, portion of each slice may be decoded by more than one segment of the video decoder mechanism. A signal director is provided for directing the decoded video image to plural video image buffers, each having plural memory locations therein, which are provided for storing a single video image. One video image buffer stores a current frame, another video image buffer stores a previous I or P video frame, and yet another buffer stores a future I or P video frame. In the case where a selected portion of each slice is manipulated, the selected portion of the slice is stored in more than one memory location. The remaining portion of each slice (the non-selected portion) is stored in only one memory location. A video output mechanism is provided for transmitting a decompressed video image. The simultaneous video decompression apparatus of the invention uses a conventional single-port memory module as buffer mechanism which, in conjunction with a signal director, to accommodate the memory access requirements that allow the decoder chips to function in a no-wait manner, i.e., there is no need for one chip to wait for the other chip to access memory. The method of the invention includes partitioning the video image into discrete slices; designating a portion of each slice to be a shared portion and designating the remainder of each slice to be an exclusive portion; defining a buffer having multiple memory locations therein; storing the image in the buffer, which includes storing the exclusive portion of each slice in a single memory location and storing the shared portion of the image in at least two memory locations; decoding, asynchronously, each slice with a discrete video decoder, wherein a video decoder reads only the exclusive portion of its assigned slice, and reads both the shared portion of its assigned slice and the shared portion of an adjacent slice to form a decoded slice; syncliromzing the decoded slices; and writing the decoded image to a video output mechanism. These and other objects and advantages will be more fully appreciated as the description which follows is read in conjunction with the drawings. Brief Descriptions of the Drawings Fig. 1 is a block diagram showing the apparatus and method of the invention. Fig. 2 depicts a single video image divided into slices. Fig. 3 depicts a video image as the image is written. Fig. 4 depicts a video image as the image is read.

Fig. 5 depicts a signal director in a write mode. Fig. 6 depicts a signal director in a read mode.

Detailed Description of the Preferred Embodiment Referring now to Fig. 1, the apparatus of the invention is depicted generally at 10. Apparatus 10, also referred to herein as a decompression apparatus, includes an input mechanism 12 which receives a compressed video input image 14. Compressed video input 14 is transmitted to a video system control, also referred to herein as a read/write control, 16, and to a video decoder mechanism, depicted generally at 18. As will be explained later herein, input mechanism 12 and system controller 16 are operable to divide video image 14 into slices. Although the thrust of this description deals with decoding video data, it will be appreciated by those of skill in the art that audio data must also be decompressed and manipulated. The term "signal data" is used herein to refer to the combined video and audio data. Decoder mechanism 18, in the preferred embodiment, includes plural video decoder chips, such as chip 1, shown at 20, chip 2 shown at 22, and chip N shown at 24. Each decoder chip represents a discrete segment of video decoder mechanism 18, and is operable to decode a slice of image 14.

Signal data to and from video decoder mechanism 18 is directed over dedicated channels 26, 28 and 30 from and to a signal director 32, respectively, which includes switches 34, 36 and 38. Channels 26, 28 and 30, referred to herein as Channel 1, Channel 2 and Channel N, respectively, allow a decoded image slice to be transmitted from a given decoder chip to and through signal director 32, without regard for the stage of decoding occurring in another decoder chip. Switches 34, 36 and 38, referred to herein as Switch 1, Switch 2 and Switch N, respectively, determine which portion of a video slice is directed over channels 40, 42 and 44, referred to herein as Channel 1', Channel 2' and Channel N¹, respectively, to a particular location in a buffer mechanism, depicted generally at 46, and also provide access by the decoder chips to the signal data stored in buffer mechanism 46. Again, the use of multiple channels instead of a common bus allows transmission of data without regard for the stage of decoding of related video and audio data in the decoding process.

In the preferred embodiment, buffer mechanism 46 includes three video image buffers (VLBs), designated VTB 1, depicted at 48, VEB 2, depicted at 50, and VIB 3, depicted at 52. Each video image buffer includes plural memory locations. As depicted, and now referring to VTB 1, memory location 1 (M 1) 48a, and memory location 2 (ML2) 48b, are depicted. ML1 and ML2 are present in each VIB. Each VTB stores a single video image. As designated herein, VLB 1 stores a current frame, VLB 2 stores a previous I or P video frame, and VTB 3 stores a future I or P video frame. As will be explained more fully later herein, a selected portion of each slice is stored in more than one memory location, i.e., the selected portion will be stored in ML1 and ML2 of each video image buffer. The remaining, or non-selected portion, of each slice is stored in only ML 1. Once the image has been properly decompressed, it is transmitted from buffer mechanism 46 to a video output mechanism, or display device, 54 which displays the video output 56. As previously noted, the video decompression apparatus of the system utilizes plural individual decoders, which work in parallel and reconstruct individual portions of the resulting video output image 56.

In the description which follows, the apparatus will be described as having only two video decoder chips, i.e., chip 1 and chip 2. Each chip decodes part of compressed video image 14, which is partitioned into as many "slices" as there are decoder chips. It should be appreciated by those of skill in the art that more than two chips may be used in practicing the method of the invention and in building the apparatus thereof, which would simply involve the division of the video image into more than two slices.

Referring now to Fig. 2, one possible technique for accomplishing an object of the invention is to divide a video image 14 into a top slice 62 and a bottom slice 64. This is accomplished by system controller 16 and input mechanism 12 and is not dependent on the compressed video input image 14.

Referring now to Fig. 3, a reconstructed image includes four areas: the top of slice 1 (58a), also referred to herein as a remaining portion, is written and read only by video decoder chip 1. The bottom portion of slice 1 , 58b, also referred to herein as a shared or boundary portion, will be written exclusively by chip 1, but is made available for reading by chips 1 and 2. In this case, the shared bottom of slice 1, 60b, is stored in both ML1 and in ML2. Likewise, the bottom of slice 2 is written and read exclusively by video decoder chip 2. The top portion of slice 2 is written by chip 2, but is read by chips 1 and 2 and is stored in both MLl (48a, 50a) and ML2 (48b, 50b). Switches 34, 36 are set to write the appropriate slice portions to VDC 1 and VDC 2, prior to decoding, primarily so that the selected, or shared, portions of slice 58 and slice 60 are written into MLl and ML2. During decoding, and now referring to Figs. 1 and 5, each decoder chip 20, 22 writes the portion of the image that it reconstructed into non-overlapping segments of MLl, 48a, and ML2, 48b, in VTB 1, which comprises the "current frame." Because the process of image reconstruction requires that motion compensation calculations, as described in the references cited earlier herein, be made, and because such calculations require access to previously stored image data, VTB 2 is used to store previous I or P video frame information for reference during reconstruction. Specifically, the reconstruction of portions of the image which are along or near the boundary separating the area being reconstructed by chip 1 and chip 2 may require that one or both of the decoder chips have access to a portion of the previous image which is stored in memory locations of VTB 2. Because the decoding process is asynchronous, two adjacent decoder chips may simultaneously require read access to the same pixels of a previous decoded image.

The simultaneous video decompression apparatus of the invention uses a conventional single-port memory module, such as a synchronous DRAM, as buffer mechanism 46 which, in conjunction with signal director 32, and dedicated channels 26, 28, 30, 40, 42 and 44, accommodates the memory access requirements that allows the decoder chips to function in a no-wait manner, i.e., there is no need for one chip to wait while another chip accesses memory. Channels 1', 2' and N\ 40, 42 and 44, respectively, provide two-way transmission of data signals between signal director 32 and buffer mechanism 46. The data signals are routed in buffer mechanism 46 to/from the appropriate VTB, as required, in a manner that allows for no-wait signal processing. As previously noted, each video image buffer contains a single video image. In the case of an MPEG compliant decoder, the previously described three video image buffers are required. During the decoding of a single MPEG2 encoded frame, decoder chips 1 and 2 write to VTB 1 to create the current video frame. Each decoder chip may read from one or both of the other video image buffers to retrieve previously-decoded reference pixel values. An individual video image buffer may be used as the current video frame during one frame interval, and as a reference video frame during the subsequent frame interval. Memory address allocation, direct hardware control or other well-known means may be used to associate a particular video image buffer with either current or reference frame usage, as described in U.S. Patent No. 5,473,379. As will be appreciated by those of skill in the art, the method and apparatus described herein may also be applied to a video decoder system which is used to decode other motion-compensation based encoding algorithms, including H.261, H.263, and MPEG1. As other video encoding algorithms, such as H.261, require only a single reference image, only two video image buffers may be required in a system designed to decode a compressed H.261 image. Within any video image buffer, each pixel is stored in MLl or ML2. In the case of a Pixel which is accessed by a single decoder chip during reconstruction, only one of the memory locations is used. In the case of a pixel which may be accessed by two decoder chips, i.e., those pixels lying within a certain distance of the boundary between the active areas of the two decoder chips, duplicate copies of the pixel data are stored in both MLl and ML2 within a video image buffer.

As depicted in Fig. 5, the arrangement of the apparatus during a write operation is depicted. When writing to VTB 1, each decoder chip writes to the memory locations which contain the non-selected and selected portions of its own image slice. When a decoder chip writes a selected portion of the image, memory locations 1 and 2 are both written to. During a read operation, a decoder chip reads from the memory location which holds the video image pixels for its own slice, as well as the memory location which holds the selected portion of the video image written by the other decoder chip. The decompressed image is output as image 56, as depicted in Figs. 1 and 3.

Referring now to Figs. 4 and 6, the arrangement of the apparatus during a read operation is depicted. As previously noted, an image is partitioned into two slices, as in the case of image 14, slices 62 and 64 are formed. The top of slice 1, 62a, is written and read only by video decoder chip 1. The bottom portion of slice 1, 62b, will be written exclusively by chip 1, but is made available for reading by chips 1 and 2. In the read mode, the shared bottom of slice 1, 62b, is stored in both MLl and in ML2. Likewise, the bottom of slice 2, 64a, is written and read exclusively by video decoder chip 2. The top portion of slice 2 is written by chip 2, but is read by chips 1 and 2 and is stored in both MLl and ML2 of the appropriate video image buffer. Switches 34, 36 are set to read the appropriate slice portions from VDC 1 and VDC 2, primarily so that the selected, or shared, portions of slice 62 and slice 64 are read from MLl and ML2. During a read operation, a decoder chip reads from the memory location which holds the video image pixels for its own slice, as well as the memory location which holds the selected portion of the video image written by the other decoder chip. Fig. 4 depicts the portions of video input image 14 with slices 62 and 64 as arranged for a read mode. A significant advantage of this invention is that any type of read/write memory may be used for the memory locations, including SRAM and DRAM. The arrangement of the memory locations may take any number of forms, however, in the preferred embodiment, the memory locations are arranged in planes, with one byte of Y (luminance) data per pixel and one byte each of Cb and Cr (blue and red color difference) for each four pixels. Signal director 32 includes memory address arbitration progr-tmming, in the form of "If... Then" statements, so that the appropriate memory location is accessed for each particular read and write operation. The required size of the shared memory locations is set by the range of the motion vectors processed by video decoding mechanism 18. Additional unused memory locations may be appended at the beginning or end of each image line to pad the memory addresses in order to simplify the construction of signal director 32.

Assuming that a video decompression system uses motion vectors in the range of +/- 64 pixels, the memory locations for the selected portions of the slice will be 64 lines high by the width of the image. Write access to any of the 64 lines of the memory location will result in the writing of duplicate data into each of the two copies of the memory location. Read access to any of the corresponding 64 lines will result in the reading of the data from the appropriate memory location. Thus, an apparatus and method for simultaneous video decompression has been disclosed. Further modifications and variations may be made to the invention without departing from the scope thereof, as defined in the appended claims.

Claims

1. An apparatus for simultaneous video decompression for use with a video display device, comprising: an input mechanism for receiving a compressed video image, which compressed video image is partitioned into slices, wherein each slice includes an exclusive portion and a shared portion, and wherein said slices are transmitted as data signals; a no-wait video decoder mechanism for decompressing, asynchronously, said compressed video image, and wherein discrete segments of said video decoder mechanism decodes a video image slice, and wherein said shared portion of each slice is decoded by more than one discrete segment of said video decoder mechanism; a signal director for directing said compressed -video image to said video decoder mechanisms; a buffer mechanism having plural video image buffers therein, each video image buffer having plural memory locations, for storing a single video image, wherein one video image buffer stores a current frame, another buffer stores a previous I or P video frame, and another buffer stores a future I or P video frame, wherein said shared portion of each slice is stored in more than one memory location, and wherein said exclusive portion of each slice is stored in only one memory location; and a video output mechanism for transmitting a decompressed video image.

2. The apparatus of claim 1 wherein said video decoder mechamsm includes discrete video decoder segments in the form of "N" video decoder chips.

3. The apparatus of claim 2 wherein said signal director includes "N" switches for directing data signals between said decoder mechanism and said buffer mechanism.

4. The apparatus of claim 1 which further includes dedicated channels between said decoder mechanism and said signal director for transmitting data signals therebetween.

5. The apparatus of claim 1 which further includes dedicated channels extending between said signal decoder and said buffer mechanism for transmitting data signals therebetween.

6. An apparatus for simultaneous video decompression for use with a video display device, comprising: an input mechanism for receiving a compressed video image, which compressed video image is partitioned into slices, wherein each slice includes an exclusive portion and a shared portion, and wherein said slices are transmitted as data signals; a no-wait video decoder mechanism, including discrete video decoder segments in the form of "N" video decoder chips, for decompressing, asynchronously, said compressed video image, and wherein discrete segments of said video decoder mechanism decodes a video image slice, and wherein said shared portion of each slice is decoded by more than one discrete segment of said video decoder mechamsm; a signal director for directing said compressed video image to said video decoder mechanisms; a buffer mechanism having plural video image buffers therein, each video image buffer having plural memory locations, for storing a single video image, wherein one video image buffer stores a current frame, another buffer stores a previous I or P video frame, and another buffer stores a future I or P video frame, wherein said shared portion of each slice is stored in more than one memory location, and wherein said exclusive portion of each slice is stored in only one memory location; wherein said signal director includes "N" switches for directing data signals between said decoder mechanism and said buffer mechanism; and a video output mechanism for transmitting a decompressed video image.

7. The apparatus of claim 6 which further includes dedicated channels between said decoder mechansim and said signal director for transmitting data signals therebetween.

8. The apparatus of claim 6 which further includes dedicated channels extending between said signal decoder and said buffer mechanism for transmitting data signals therebetween.

9. A method of simultaneously decompressing a compressed video image, comprising: partitioning the video image into discrete slices; designating a portion of each slice to be a shared portion and designating the remainder of each slice to be an exclusive portion; defining a buffer having multiple memory locations therein; storing the image in the buffer, which includes storing the exclusive portion of each shce in a single memory location and storing the shared portion of the image in at least two memory locations; decoding, asynchronously, each slice with a discrete video decoder, wherein a video decoder reads only the exclusive portion of its assigned slice, and reads both the shared portion of its assigned slice and the shared portion of an adjacent slice to form a decoded slice; synchronizing the decoded slices; and writing the decoded image to a video output mechanism.