JP2002502159A - Method and apparatus for encoding and decoding high performance television signals - Google Patents

Abstract

(57) Abstract: Encode and decode high performance television signals using a standard MPEG-2 compression engine and maintain the efficiency of such compression engines. SOLUTION: The architecture has parallel processing and uses a standard MPEG-2 compression engine configured in an overlapping manner so as not to degrade the compression performance. The video encoder includes a multi-region processor for encoding a continuous input of video images. Each video image is divided into regions having overlapping portions, and each processor encodes a particular region of the current video image in the sequence. Each of the area processors stores, in a local memory, a reference frame based on a preceding video image in a continuous image, and uses the reference frame for motion compensation in the encoding process. A reference frame processor connected to the plurality of local memories updates each reference frame using information from the reference frame stored in the adjacent local memory. The coded video images constitute macroblocks, and each region processor comprises means for removing specific macroblocks from the coded video image corresponding to the overlap, and the video images of the other region processors And the above encoded video image are concatenated to generate an output video signal.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION

The FCC (Federal Communications Commission) has been using ATSC digital television (Advanced Television Systems Co.) for terrestrial broadcasting.
mmittee Digital Television) Standard main elements were adopted. The ATSC digital television (DTV) standard supports five major components of the model system and delivers multimedia information to users. ITU-R (Inte
A block diagram of such a model system, as defined by task group 11/3 of the National Telecommunications Union, Radio Communication Sector) is shown in FIG. 1 and includes video, audio, transport, radio frequency / transmission and receiver. Including components. The video subsystem 100 complies with ISO / IEC IS 1
3818-2 International Standard (1994) Moving Picture Experts Group, MPEG-
2 Compresses the unprocessed video into a data element signal sequence according to the MPEG-2 standard specified in Video. The audio subsystem 102 converts the unprocessed video to the Au in the ITU.
DAC-3 (Digital Audio Compressio) specified by dio Specialist Group
n 3) Compress to digital audio data element signal sequence according to the standard.

The service multiplex and transport component 104 converts a video data element signal sequence, an audio data element signal sequence, an auxiliary and control data element signal sequence into an ISO / IEC IS 13818-2 International Standard (1994), M
It is multiplexed into a single bit signal sequence using the transport stream syntax specified in the PEG-2 System. Radio frequency / transmission component 106 includes a channel coder and modulator. The channel coder provides additional information to the transport signal sequence to enable the receiver 108 to repair a partially corrupted bit sequence . The modulator encodes the digital data into a radio frequency signal using vestigial sideband transmission.

[0003] The MPEG-2 standard utilizes five compression methods to achieve high compression ratios. These are the discrete cosine transform (DCT), differential coding, quantization, entropy coding, and motion compensation.

[0004] The DCT is applied to an 8x8 pixel block to create 64 coefficients that represent spatial frequency. For blocks with fewer pixels, the value of the high frequency coefficient is smaller,
Can also be set to zero.

[0005] A video frame is an intra-frame (independent of information from other frames).
(I frame) to restore the current frame, and P and B, which are interframes that depend on information from other frames, to restore the current frame. The P frame depends on the preceding P or I frame, while the B frame relies on the preceding I or P and the subsequent I or P to restore the current frame. These preceding or succeeding I and P frames are called reference frames. P and B frames include only the difference between the current frame and adjacent frames. For slow motion video sequences, the information content of the P and B frames is very small.

[0006] The MPEG-2 compression algorithm performs motion prediction between adjacent frames and improves the prediction capability between frames. The compression algorithm searches all four blocks, known as macroblocks, for a motion vector that provides the distance and direction of motion for the current macroblock.

[0007] The DCT (discrete cosine transform) coefficients of each block are weighted and quantized based on a quantization matrix that matches the response of the human eye. The result is combined with the motion vector and then encoded using variable length coding to generate a signal sequence for transmission.

[0008] The amount of computation required to perform the video compression specified in the MPEG-2 standard is very large. In applications that require real-time compression, such as live broadcasting, it is difficult to achieve video compression. The method of performing MPEG-2 compression includes slice-
Base and macroblock-base are known. In the slice-based method shown in FIG. 2, the video frame 120 is divided into several adjacent regions 120A. Each area is assigned to a different processor (P1, P2, P3, P4, P5) and processed. A dedicated central processor 122 manages the overall compression operation. In the macroblock-based approach shown in FIG. 3, each macroblock 124 is completely processed and passed to an output buffer 126 before processing the next macroblock.

The MPEG-2 standard defines parameter values called levels (eg, image size,
It defines an algorithm tool known as a combination of profiles and constraints on bit rate. The above known MPEG-2 compression engines are designed to satisfy the main profile in the standard main-level part for conventional broadcast television signals such as NTSC and PAL. The main level is defined as 720 pixels × 480 active lines at 30 frames per second. On the other hand, the DTV signal is composed of 1970 pixels × 1 at 30 frames per second.
080 active line. This is known as MPEG-2 High Level. The amount of computation required for a DTV signal defined as a main profile at a high level is about six times that required for a current standard television signal defined as a main profile at a main level.

[0010]

SUMMARY OF THE INVENTION

It is desirable to encode high resolution video signals while taking advantage of existing MPEG-2 compression engines, while maintaining the compression efficiency of such compression engines.

[0011] The method and apparatus of the present invention uses a standard MPEG-2 compression engine that operates with a main profile in a main level mode, and uses a standard MPEG-2 compression engine to operate at high levels, such as a DTV signal that conforms to the MPEG-2 main profile. Provide an architecture that can handle the required amount of computation. The present invention provides parallel processing using such a standard MPEG-2 compression engine as an overlapping configuration without sacrificing compression performance.

Accordingly, the video encoder of the present invention includes a plurality of domain processors for encoding an input signal sequence of a video image. Each video image is divided into regions with overlapping portions, and each processor determines a specific one of the current video image in the signal stream according to an encoding process that includes motion compensation such as an MPEG-2 main profile at the main level. Encode the region. Each of the area processors stores a reference frame in a local memory based on a preceding video image in the signal sequence and uses the reference frame for motion compensation in the encoding process. A reference frame processor connected to the plurality of local memories updates each reference frame with information from the reference frame stored in an adjacent local memory. The encoded image is composed of macroblocks, and each region processor has means for removing specific macroblocks from the encoded video image corresponding to the overlap, and the resulting code An output video signal sequence is generated by concatenating the segmented video image with a video image of another area processor.

In one embodiment, each region processor includes an image selection unit that selects a particular image region from each of the video images. The compression engine compresses the selected image area to generate a signal sequence of the compressed area of the macroblock. The macroblock removing device removes a specific macroblock from a signal sequence in a compressed image area corresponding to an overlapped portion. The signal sequence concatenation unit concatenates the signal sequence of the compressed image region with the signal sequence from the processor in each region to generate a signal sequence of an output video.

Although the preferred embodiment has multiple region processors to handle overlapping regions, the present invention encompasses a single processor embodiment that processes each region sequentially.

According to another configuration of the present invention, a video decoder includes a demultiplexer, a plurality of area decoders, a reference frame memory, and a multiplexer. The demultiplexer demultiplexes the compressed video image signal sequence into a plurality of region signal sequences. Each video image is divided into contiguous regions, and the signal sequence in each region is associated with a particular region. Each of the area decoders decodes a signal sequence of a specific area according to a decoding process including motion compensation such as an MPEG-2 main profile at a main level. The reference frame memory stores a reference frame associated with each area decoder. The area decoder takes in the reference frame of the adjacent area and uses it for the motion compensation processing. The multiplexer multiplexes the signal sequence of the decoded area into the decoded output signal sequence.

The above and other objects, configurations and advantages of the present invention will become apparent from the following more detailed description of the preferred embodiments of the present invention shown in the accompanying drawings. In the drawings, the same reference numerals indicate the same parts in different drawings.

DETAILED DESCRIPTION OF THE INVENTION The present invention relates to a known MPE in a main profile (mp / ml) compression engine.
Adopting a parallel processing configuration with the advantages of the G-2 main profile, providing a high performance compression engine and conforming to the MPEG-2 main profile at a higher level
Encode a high resolution television signal such as a TV signal. FIG. 4 illustrates a first method of using the MPEG-2 compression engine in a parallel configuration. This configuration has a total of 9
It consists of two MPEG-2 mp / ml compression engines and processes adjacent areas containing ATSC DTV video images (1920 pixels x 1080 lines).
Each MPEG-2 mp / ml engine uses NTSC video streaks (720 pixels x
480 lines). As shown in FIG. 4, engines 3, 6, 7, 8 and 9 encode regions smaller than NTSC images.

The compression provided by this first method is less than optimal. The motion compensation performed within each engine is naturally constrained to not search beyond its NTSC image format boundaries. As a result, macroblocks along the boundaries between assigned engine regions do not always receive the effect of motion compensation.

The preferred method of the present invention shown in FIG. 5 comprises a parallel arrangement of MPEG-2 compression engines, which process the overlapping areas 146, 148, 150, 152 of the ATSC DTV video image 142. It is arranged to be. In the preferred method, the motion compensation performed by the particular engine for that particular region is extended to adjacent regions. As described in the Background of the Invention, motion compensation uses a reference image (I or P frame) to predict the current frame in the frame encoding process. In a preferred method, the motion compensation is extended to adjacent regions,
At the end of the frame encoding process, the reference image is updated with information from the reference frame of the adjacent engine.

More specifically, each engine stores up to two reference frames in memory. At the end of the frame encoding process, if either of the two reference frames is updated, the reference frame will be further updated to reflect the frame encoding results from the adjacent engine.

FIG. 6 shows a preferred embodiment of the video encoder 100 A of the present invention. Video encoder 100A includes digital video receiver 160 and compression engine subsystem 162. Digital video receiver 160 accepts uncompressed video from an external source in one of two different digital input formats: Panasonic 720P (sequential scanning) parallel format and 1080
One of the serial formats. The 1080I serial format is SMPT
It has uncompressed 1080 line interlaced (1080I) video at 1.484 Gbps speed according to the E292M standard. Digital receiver 160 converts the input signal to a common internal format called the TCI422-40 format, in which 20 bits translate two Y-components with a 10-bit resolution and 20 bits translate a 10-bit resolution. It has a color component.

In the preferred embodiment of the video compression engine subsystem 162 shown in FIG. 7, including FIGS. 7A-7C, a video input connector 200, a system manager 202, a bit allocation processor 204, some domain processors 206 and a PES
A header generator 208 is included. Although nine domain processors 206 are shown in the configuration of FIG. 7, other configurations are possible, for example, a twelve domain configuration could be implemented to provide a greater range of motion compensation. It is. Each region processor 206 includes a local image selection unit 210, an MPEG-2 compression unit 212,
It includes a macroblock eliminator and a concatenation unit 214/216 for signal sequences and a local memory 218. Compression subsystem 162 also includes one or more reference image managers (RIMs) 220. In the configuration of FIG.
There is an IM220. The RIM 220 will be described in more detail.

The MPEG-2 compression unit 212 is preferably an IBM Model MM30 single package, three-chip unit, but any standard MPEG-2 compression engine capable of main profile at main level operation may be used.

The video input connector 200 is connected to the system control bus 222 and the TCI
A video data bus 224 called a 422-40 bus is terminated. The control bus 222 transmits control data from a system control processor (not shown) to the system manager 202. TCI422-40 bus 2
24 transmits video data from the digital receiver 160 (FIG. 6).

The system manager 202 initializes the area processor 206, interacts with external devices, monitors the video processing status, synchronizes the processors, and
Update S. This function is performed using an Advanced Micro Devices AM29K chip.

The system manager 202 stores all executable FPGA files in the internal FLAS
H memory. After startup, the system manager initializes all processors and FPGAs with pre-allocated files from FLASH memory. The system manager 202 sets the following parameters for the MPEG-2 compression unit 212. GOP structure Frame sheet Sequential encoding for 720P video Interlaced encoding for 1080I video Frame size to encode Table 1 below shows each MPEG-2 compression unit 212 for 1080I encoding
Indicates the frame size.

[0027]

[Table 1]

Table 2 below shows the frame size of each MPEG-2 compression unit for 720P encoding.

[0029]

[Table 2]

The system manager monitors the video compression process. The manager polls the normal status registers in the local image selection unit, the MPEG-2 unit, the macroblock removal unit and the signal string concatenation unit of each area processor 206 at a rate of once per second.

The system manager 202 synchronizes the frame coding processing of the entire nine area processors 206. The following shows the motivating factors behind the need for synchronization. Next, a task requested by the system manager to synchronize the parallel processors will be described.

The scalable MPEG-2 architecture requires that each region processor 206 finish the current frame encoding process before the start of the next frame. This requirement arises from the need to update reference images across adjacent parallel processors. Each MPEG-2 engine uses the internal reference image to compress the current image. The internal reference image is derived from the result of the compression processing on the preceding frame. In the scalable MPEG-2 architecture of the present invention,
The portion of the reference image is updated using a reference image from an adjacent processor.

The following items give rise to the need for synchronization. 1. The reference image is updated after each encoding process. 2. Each MPEG-2 compression unit needs to update the internal reference image using the information of the reference image in the adjacent processor, and then correctly encode the next image.

According to FIG. 8, each MPEG-2 compression unit generates a current image compression completion (CICC) signal 250 after each encoding process. Upon detecting all CICC signals, the system manager 202 triggers the reference image manager 220 to update the internal reference image of each MPEG-2 compression unit using a common reference image update (RIU) signal 252. When all CICC signals are active, the system manager needs to respond immediately. This is because the delay affects the MPEG-2 engine encoding time.

When the update is completed, each reference image manager updates the reference image (RIUC)
The signal 254 is activated. Upon detecting all RIUC signals, the system manager triggers all local image selection units 210 and begins loading the next frame into the compression unit 212 via a common start image loading (SIL) signal 256. . The delay between the time when the RIU becomes active and the time when the RIUC becomes active may be as long as one cycle. When all RIUC signals become active, the system manager needs to respond immediately.

[0036] The system manager updates the PTS in the PES header generator. The system manager is interrupted each time the region processor # 1 receives a new screen header. The system manager then calculates the new PTS value from the STC value latched at the video input port of processor # 1 ', and the frame type from the compressed output port of processor # 1'.

The bit allocation processor 204 operates to ensure that the cumulative compression bit rate from all domain processors matches the externally defined target compressed video bit rate. The bit allocation processor dynamically changes the compression quality setting of each MPEG-2 engine to maintain optimal video quality.

The local image selection unit (LISU) 210 extracts a local image from uncompressed input data on the TCI 422-40 data bus 224. The unit is MPE
The local image is output in a format that matches the input format defined by the G-2 unit 212. The LISU supports the following programmable registers. 1. Input Video Format Register: This register defines the video format represented by the data on the TCI422-40 bus. 0 = 1080I, 1-720P Local image position registers: These registers specify the position of the local field image in the entire field image 300 (FIG. 9).

The register specifies a point in the field image, but does not specify a reconstructed sequential image. Note that the 720I video has only one field image per frame, while the 1080I has two field images per frame.

The four registers specify the corner positions of the local image 302 in the whole image 300 as shown in FIG. The four registers specify: Hstart register: Pixel index of the first active pixel of local image 302. The first pixel in the whole image 300 has an index value of 1. Hstop register: Pixel index of the first non-active pixel after the local image. Vstart register: Line index of the first active line in the local image. The first line of the whole image has an index value of one. Vstop register: line index of the first non-active line after the local image

Table 3 below shows the register values for the various formats supported by each MPEG-2 unit 212.

[0042]

[Table 3]

The macroblock removal and bit concatenation (MRBC) unit 214/216 is used to transmit M bits in the main level bit signal sequence received from the MPEG-2 unit 212.
It has a role of converting the PEG-2 main profile into an ATSC compatible bit signal sequence. Each MRBC unit performs two functions of macroblock removal and bit concatenation by scanning and editing the bit signal sequence from the MPEG-2 unit 212 and communicating with other MRBC units.

The scalable MPEG-2 architecture (FIG. 7) provides nine MPEG-2 compression units 212 for 1080I video format encoding and 720P
Eight MPEG-2 compression units are used for video format encoding.

Each MPEG-2 compression unit has a target image 3 called an active area 310.
It serves to compress a specific area of 00. The target image 300 is covered by an active area 310 without overlap. FIG. 10 shows the pre-processing area 310B and the active area 310 for 1080I.

Each MPEG-2 compression unit compresses an area (pre-processing area) larger than the active area of the target image 300. The active area 310 is a sub-area of the corresponding pre-processing area 310B. Thus, the target image is also covered by the pre-processing area, but has overlap between adjacent pre-processing areas. All pre-processing areas 31
0B, the active area 310 or the overlap area 310A is always 16 (
Or 32), the active area can be obtained by removing some macroblocks from the pre-processing area.

The macroblock removal unit 214 removes macroblocks that are in the overlap area 310 A but not in the functional area 310.

The size of the functional area is determined from the following. Hact1 = 592 Vact1 = Vact3 = 352 Hact2 = 480 Vact2 = 128 Hact3 = 608 Vol = 128 Hol12 = 128 Hol23 = 112

Referring now to FIG. 11, the following integer parameters are defined for each MPEG-2 compression unit 212 with respect to macroblock position. raw height: height of the pre-processing area 310B. raw width: width of the pre-processing area 310B. left alignment: active area 310 macroblock 320
The mark at which starts horizontally, and therefore the macroblock to the left of this mark in the pre-processing area, needs to be removed. right alignment: The mark at which the active macroblock ends horizontally, and therefore the macroblock to the right of this mark in the pre-processing area, needs to be removed. top alignment: The mark at which the active area macroblock starts in the vertical direction, so that the macroblock above this mark in the pre-processing area needs to be removed. bottom alignment: The mark at which the active area macroblock ends vertically, and therefore the macroblock under this mark in the pre-processing area, needs to be removed.

The following definitions of the other two parameters are specified for 1080I. head mb skip: left Number of macroblocks skipped from the alignment and not skipped in the active area. tail mb skip: right from the last non-skippable macroblock in the active area Number of macroblocks skipped to alignment.

For convenience of expression, the following parameters: ((raw width, raw heigh ht), (left alignment, right alignment), (
top alignment, bottom The following convention is used to represent the alignment)) and is called the configuration vector for the MRBC unit. This configuration vector corresponds to the current MPEG-
Care must be taken to define the boundaries of the pre-processing area 310B and the active area 310 of the two-compression unit, and thus define which macroblocks should be deleted and which macroblocks should be preserved.

The value of the configuration vector of each MRBC unit for 1080I is as follows. ♯1: ((45, 30)), (0, 41), (0, 26)) ♯2: ((45, 30)), (4, 41), (0, 26)) ♯3: ( (45, 30)), (3, 45), (0, 26)) ♯4: ((45, 24)), (0, 41), (4, 20)) ♯5: ((45, 24) )), (4, 41), (4, 20)) ♯6: ((45, 24)), (3, 45), (4, 20)) ♯7: ((45, 30)), ( 0, 41), (4, 30)) ♯8: ((45, 30)), (4, 41), (4, 30)) ♯9: ((45, 30)), (3, 45) , (4, 40))

The MRBC unit 214/216 scans and edits the coded bit signal sequence for the unprocessed base slice. In the horizontal direction, raw top and top of region Bottom between alignment It is necessary to delete the macroblock in the region between the alignment and the lower end of the pre-processing region. raw r
raw for each column of the egion left end start and left of region a
between light and right alignment and raw regi
Macroblocks in the area between the right end of on and need to be deleted. The resulting bit signal sequence is called mr-processed row. Each M
The PEG-2 unit uses a single slice for each column, so mr-pr
The processed row is also mr-processed sl in this sense.
Also called ice.

For 1081I, some of the macroblock removal units besides forming the mr-processed row, mb skip and / or tail mb It is necessary to form a value for skip. In the beginning,
Local variable quant quantizer using trace scal
e Record the code. This quantiser scale code is initially the quantifier in the slice header scale code and left Quantizer by alignment scale co
Updated each time de occurs in the next macroblock.

Starting from a particular slice, the macroblock removal unit scans the coded bit signal sequence and performs the following procedure. -Head as required mb Calculate the skip (specific to 1081I). -Update quant trace until Left alignment. m acroblock Check that the quant flag is set for the first non-skipping macroblock in the active area. Macroblock if not configured The quant flag is set and the quantifier scale code value is quant The value of trace is set and the macroblock header is re-formed accordingly (specific to MRBC units in columns 2 and 3 for 1081I encoding). -Only the macroblocks held in the active area in the scanning process form mr-processed slices. -Tail as needed mb Calculate the value of skip (specific for 1081I).

The bit stream from each local MPEG-2 compression unit 212 needs to be concatenated to generate a uniform ATSC compliant DTV bit stream. Each MRBC unit needs to place its local mr-processed slice in the output buffer 208 at the correct time (FIG. 7). In other words, the MRBC unit 21
The operation of 4/216 needs to be synchronized. Thus, a token mechanism is used to synchronize the MRBC units.

FIG. 12 shows a communication model for 1080I video format encoding. For 720p video format encoding, processor # 2
, $ 5, $ 8 are excluded as shown in FIG. Additional rows are added at the bottom.

The token indicates that the MRBC unit having the token can send the signal sequence of the bit to the output buffer via the output bus 228. When an MRBC unit receives a completion signal from another MRBC unit, that signal has a token. MRBC unit # 1 is responsible for starting a new token. M
RBC unit # 1 has a timeout variable. If a timeout occurs, an error occurs and the system manager is reset. The token is sent over a designated line 270 between the MRBC units. Only one valid token is allowed at a time.

For 1080I video format encoding, each DTV slice is obtained by concatenating three local slices in three MRBC units in the same column. Each macroblock header contains information about the number of skipped macroblocks between the preceding non-skipped macroblock and the current macroblock, and this information is updated when a local mr-processed slice is incorporated into a DTV slice. There is a need. Further, the first non-skipped macroblock in the second and third locally processed slices needs to have its header updated.

The correct header information is inserted into the DTV bit signal sequence by the MRBC unit. The header information is obtained by scanning a bit signal sequence from the output buffer of the local MPEG-2 unit 212. For 1081I video format encoding, MRBC units # 4 and # 7 only have the role of inserting slice header information.

The macroblock skip information from the last MRBC unit, tail mb
 The skip is accepted and the local head mb combined with skip
. All macroblock skip information is next in the mr-processed slice.
Is inserted into the macroblock header of the first non-skipped macroblock of
The bit signal sequence is then placed in a DTV output buffer. Next, local tail  mb The skip connects to the next MRBC unit via a dedicated 8-pin data bus 228.
Sent. The MRBC unit of the first row only (# 1, $ 4, $ 7) is tail  mb The skip information is sent out, and the MRBC unit (# 2, # 5, #) in the second row is sent.
8) tail mb both receiving and sending skip information,
The MRBC units in the third row ($ 3, $ 6, $ 9) are tail mb skip
Receive information only.

Upon receiving the token signal, the MRBC unit updates the mr-processed slice, outputs this to the DTV output buffer, then activates the token line, and passes the token to the next MRBC unit.

The next step of sending the token for 1081I video format encoding is:
Follow the rules below. -If the next MRBC unit exists in the same column, send the token to the next MRBC unit. -If the current MRBC unit is at the end of a row of MRBC units, the token is sent to the first MRBC unit in the same row unless the slice sent to the output buffer is the last slice in the active area. When the slice sent is the last slice in the active area, the token is sent to the first MRBC in the next column. The only exception is that MRBC unit # 9 sends a token to MRBC unit # 1 after the last mr-processed slice.

As described above, the reference image manager (RIM) 220 (FIG. 7)
Manage the updating of the reference image used by each of the compression units 212. Each RI
The M220 transmits information from the local memory 218 in one area processor 206 to the local memory of the adjacent processor. The reference image in each MPEG-2 unit is updated by the compression engine during the frame encoding process. The reference frame is updated only when encoding an I or P frame. The following example shows the order in which two reference frames are updated as shown in FIG.

Consider two reference images stored in reference buffer A and reference buffer B of local memory 218 and a compressed output sequence IBPPBBPBBBIBB. A reference image is formed by a compression process on the first I frame. This image is stored in the reference buffer A. The compression process for the next two B frames is
No reference image is formed. The compression process for the next P frame forms a reference image. No new reference image is formed until the next P frame. The reference image of the P frame is stored in the reference buffer A. The preceding reference image generated when compressing the I-frame is then erased.

The reference buffer only has RI reference if it is modified by the frame compression process.
It needs to be updated by M220. The reference image 400A is updated by the RIM 220 using the information of the reference image 400A from the adjacent processor, FIG.
Then, the area 400C of the reference image 400A in the center processor is updated.
The area 400B in the reference image of the adjacent processor used in M is shown. For corner processors that do not have adjacent processors on these planes and on all planes, the regions bordering the empty neighbors in the reference image need not be updated.

The RIM 220 maintains a relationship between frame types and reference image updates based on the above guidelines. The RIM identifies which of the two reference buffers A and B (if any) were updated by the MPEG-2 unit at the end of each frame encoding process.

The RIM 220 is a Henc from the IBM encoder By monitoring the inc signal, the output time of the screen information block, and the vertical synchronization signal of the video input port, the time when the MPEG-2 unit has completed encoding the current frame is determined.

RIM is an MPEG-2 compression unit, Henc The start address in the local memory for the color and luminance reference image is calculated using the inc signal and the information from the screen information block extracted from the update state of the reference buffers A and B. It can be assumed that the luminance and color reference start addresses do not change during each frame encoding process. The start address is defined by the compression configuration.

The RIM 220 updates all changed reference images at the end of each encoding process. The RIM updates each reference image based on the following table, as also shown in FIG.

[0071]

[Table 4]

-BR1 specifies the first pixel immediately after the regions Rtln and Rbln. -AR1 specifies the first pixel immediately after the regions Atln and Abn. -BRC specifies the line immediately after the region Rtn. -ARt specifies the line immediately after the area Atn. -ARb designates the line immediately after the regions Aln and Arn. -BRb specifies the line immediately after the regions Rln and Rrn. -BRr specifies the first pixel immediately after regions Rbn and Rtn. -ARr specifies the first pixel immediately after the regions Abn and Atn.

The four RIM processors 220 perform the functions necessary for the preferred embodiment of FIG. The components within each RIM processor are shown in the block diagram of FIG. A single RIM processor 220 now has Ptl, Ptr, Pbo
And the local memory of the four video processors, denoted as Pbr. R
The IM has access to 64-bit data, 9-bit addresses and associated DPRAM control signals. Within the RIM, 12 local managers 22
OA manages another image area 400B (FIG. 15) around each reference image 400.

FIG. 18 shows components within each local manager 220A. The local manager holds four buffers 220B, 220C, two of which hold the border image for reference image A and the other two hold the border image for reference image B. To operate data in the AR area during each frame encoding process,
The MPEG-2 unit reads and writes to local memory 218 (FIG. 7). At the same time, the local manager sends the AR / BR buffers 220B, 220
Update one of C. In this example, the buffer A holds data for mirroring the AR area in the local memory of the MPEG-2 unit. At the end of the frame encoding process, the controller 232 in the local manager
Buffer A is remapped to the BR area of the adjacent MPEG-2 unit. Neighbor M
The buffer B mapped as the BR area of the PEG-2 unit is remapped to the AR area of the center processor. It is the responsibility of the controller to remap the buffer each time the reference image is updated through the frame encoding process.

The PES header generator 208 inserts the PES header generator into a video element signal sequence (simply a signal sequence). The PES header generator extracts this information directly from the compressed signal sequence. For screen type information, the generator extracts this information from the screen header in the compressed bit signal sequence. For PTS values, the PES generator converts the PTS value to screen type pt, gop structure go
It is calculated from ps and STCVi information of the input video STC time stamp.
The PES header generator latches the STCVi value using the vertical synchronization signal entering the MPEG-2 unit of the area processor # 1.

The amount of calculation required to decode a signal sequence of an ATSC compliant video is very large. Thus, the common sequential decoding architecture used for decoding standard MPEG-2 ml in mp does not meet the requirements. The scalable architecture of the present invention decodes an ATSC DTV video signal sequence using an existing MPEG2 decoding engine.

[0077] Embodiments of the decoder system include four parallel region decoders. The system requires that each decoder be able to decode 2.5 times the video frame size of the NTSC format. This requirement is not unreasonable because it is not a severe requirement for known decoding algorithms. As shown in FIG. 19, each of the decoders has a local area 500A, 500B, 500C, 5C of 1920 pixels × 270 lines in an ATSC frame 500 that is 1920 pixels × 1080 lines.
00D is decoded.

FIG. 20 is a block diagram of a decoder system 520 including a demultiplexer 522 for compressed signal streams, four parallel domain decoders 524 A, 524 B, 524 C, 524 D, reference frame stores 526 A, 526 B, 526 C and a multiplexer 528. Indicates a component.

The demultiplexer 522 of the compressed signal sequence demultiplexes the compressed video signal sequence 521 based on the slice header information to form the region signal sequences 523A and 523.
B, 523C and 523D. Area decoders 524A, 524B, 524
C and 524D decode the compressed bit signal sequences 523A, 523B, 523C and 523D that are fully compatible with MPEG-2 with the following two exceptions. The exception is that the signal train limits the frame to 1920 pixels x 270 lines and the motion vector extends beyond the vertical dimension up to a maximum of 270 lines. In a preferred embodiment, the existing MPEG-2 decoder is modified to
Satisfy the exceptions specified for 24A, 524B, 524C, 524D.

The region decoder also needs to deal with decoding dependencies between adjacent regions. There is one important dependency on the decoding process: motion vector compensation. Motion vector compensation creates pixels in the current macroblock by utilizing pixel information from a reference image. The reference image is an image made from the preceding I or P frame. As shown in FIG. 21, through motion compensation, the procedure reaches a region beyond the local region and creates a pixel in the current block. The maximum depth reached by the procedure is determined by the maximum length defined by the motion vector.

Each region decoder makes the reference image available to other decoders so that the other decoders correctly execute the motion vector compensation procedure. The reference image is shared by the adjacent areas. The precondition is that the maximum motion vector does not exceed the width of each region of 270 lines. This precondition is justified because the actual compressed video sequence does not form a motion vector larger than 270 lines.

The reference image is shared through reference image storage as shown in FIG. The region decoder writes to two memory locations 530, 532, 534, 536 simultaneously. One (530, 536) is for future access by the current decoder, and the other (532, 534) is for future access by adjacent decoders. In the embodiment, when the motion vector compensation routine is executed, simultaneous reading is executed by two decoders.

The multiplexer 528 multiplexes the uncompressed frame area and returns to the entire frame. The multiplexer forms an 8-bit digital data signal sequence according to the SMPTE 274M standard.

[0084]

[Equivalent]

While the invention has been illustrated and described in detail with reference to preferred embodiments, workers skilled in the art will recognize that various modifications can be made in form and detail without departing from the spirit and scope of the invention as defined by the appended claims. Is understandable. Those skilled in the art will recognize, and be able to ascertain using no more than routine experimentation, many equivalents to the specific embodiments of the invention specifically described herein. Such equivalents are intended to be included within the scope of the claims.

[Brief description of the drawings]

FIG. 1 is a block diagram of one model of a modern television system.

FIG. 2 is a block diagram illustrating a slice-based compression method for an MPEG-2 main profile at the main level.

FIG. 3 is a block diagram illustrating a macroblock-based compression method for an MPEG-2 main profile at the main level.

FIG. 4 is a diagram showing a first processor configuration according to the present invention.

FIG. 5 illustrates a preferred processor configuration according to the present invention.

FIG. 6 is a block diagram of the video encoding subsystem of the present invention.

FIG. 7 is a schematic block diagram of a video compression engine of the video subsystem of FIG. 6;

FIG. 7A is a schematic block diagram illustrating a portion of the video compression engine of FIG. 7;

FIG. 7B is a schematic block diagram illustrating a portion of the video compression engine of FIG. 7;

FIG. 7C is a schematic block diagram illustrating a portion of the video compression engine of FIG. 7;

FIG. 8 is a block diagram showing a synchronization configuration for the compression engine of FIG. 7;

FIG. 9 is a diagram illustrating image selection of a part from the whole image for the engine of FIGS. 7A to 7C.

FIG. 10 is a diagram showing a pre-processing and execution region of the entire image of FIG. 9;

11 is a diagram showing an execution area in a pre-processing area of the image of FIG. 10;

FIG. 12 is a block diagram of a token passing scheme for a 1080I video processing configuration.

FIG. 13 is a block diagram of a token passing scheme for a 720p video processing configuration.

FIG. 14 is a block diagram showing allocation of reference images in a reference buffer of a local memory of the system of FIG. 7;

FIG. 15 is a diagram showing a reference image updating configuration according to the present invention.

FIG. 16 is a diagram showing an area of a reference image according to the present invention.

FIG. 17 is a block diagram of a reference image manager of the system of FIG. 7;

18 is a block diagram of a local manager of the reference image manager of FIG.

FIG. 19 is a diagram showing a decoding configuration of the present invention.

FIG. 20 is a block diagram of the decoder system of the present invention.

21 is a diagram illustrating motion compensation in the decoder system of FIG.

FIG. 22 is a block diagram of a reference frame store of the decoder system of FIG. 20;

──────────────────────────────────────────────────続 き Continuation of front page (81) Designated country EP (AT, BE, CH, CY, DE, DK, ES, FI, FR, GB, GR, IE, IT, LU, MC, NL, PT, SE ), OA (BF, BJ, CF, CG, CI, CM, GA, GN, GW, ML, MR, NE, SN, TD, TG), AP (GH, GM, KE, LS, MW, SD, SZ, UG, ZW), EA (AM, AZ, BY, KG, KZ, MD, RU, TJ, TM), AL, AM, AT, AU, AZ, BA, BB, BG, BR, BY, CA, CH, CN, CU, CZ, DE, DK, EE, ES, FI, GB, GD, GE, GH, GM, HR, HU, ID, IL, IN, IS, JP, KE , KG, KP, KR, KZ, LC, LK, LR, LS, LT, LU, LV, MD, MG, MK, MN, MW, MX, NO, NZ, PL, PT, RO, RU, SD, SE, SG, SI, SK, SL, TJ, TM, TR, TT, UA, UG, UZ, VN, YU, ZWF term (reference) 5C059 KK13 LC09 MA00 MA05 MA23 MC11 ME01 NN11 PP05 PP06 PP07 SS02 SS05 TA13 TA30 TB06 TC25 TC32 TC36 TC42 TC44 UA02 UA05 UA33 UA37 [Continued from the abstract] The output video signal is generated by concatenating the video image of another area processor and the previously encoded video image.

Claims (27)

[Claims] 1. A processor for encoding a signal sequence at an input of a video image, wherein each video image is divided into regions having overlapping portions, the processor comprising: A processor that encodes each region and stores a reference frame in a memory based on a video image signal sequence preceding the region and uses the reference frame for motion compensation of an encoding process; and a reference frame connected to the memory and adjacent to the memory. A reference frame processor for updating each reference frame with information from the video encoder. 2. The method of claim 1, wherein the processor comprises a plurality of region processors, each of which is assigned to encode a particular region, and wherein the memory comprises a local memory of each region processor. Video encoder. 3. The encoded video image of claim 2, wherein the encoded video image comprises macroblocks, and each of the region processors removes a specific macroblock from the encoded video image corresponding to the overlapped portion. Means for generating an output video signal sequence by concatenating the obtained encoded video image with the encoded video image of another area processor. 4. The encoded video image of claim 1, wherein the encoded video image comprises macroblocks, and wherein the processor removes specific macroblocks from the encoded video image corresponding to the overlapped portion. A video encoder for generating an output video signal sequence. 5. The video encoder according to claim 1, wherein said encoding process for each region comprises MPEG-2 encoding having a main profile at a main level. 6. The video encoder according to claim 1, wherein the input signal sequence is an ATSC-compliant digital video signal sequence. 7. The method according to claim 1, wherein the input signal sequence has a high level of MPEG-2.
A video encoder that is a digital video signal sequence that conforms to the main profile. 8. A plurality of regional processors for encoding an input signal sequence of a video image, wherein each video image is divided into regions having overlapping portions, each processor including a current video including motion compensation. A plurality of area processors that encode a specific area of an image and store a reference frame in a memory based on a video image signal sequence preceding the same and use it for motion compensation of an encoding process; and connected to the plurality of local memories. And a reference frame processor that updates each reference frame using information from a reference frame in an adjacent area. 9. The encoded video image of claim 8, wherein the encoded video image comprises macroblocks, and each of the region processors further removes a particular macroblock from the encoded video image corresponding to the overlap. And a means for generating an output video signal sequence by concatenating the obtained encoded video image with the encoded video image of another area processor. 10. The video encoder according to claim 8, wherein said encoding process for each region comprises MPEG-2 encoding having a main profile at a main level. 11. The video encoder according to claim 8, wherein the input signal sequence is an ATSC-compliant digital video signal sequence. 12. The video encoder according to claim 8, wherein the input signal sequence is a digital video signal sequence conforming to an MPEG-2 main profile at a high level. 13. An input signal sequence of a video image group in which each video image is divided into regions having an overlap portion is generated, and for each region, the input signal sequence in the signal sequence is subjected to an encoding process including motion compensation. Encode a specific region of the current video image, store a reference frame in a local memory based on the previous video image in the signal sequence and use it for motion compensation in the encoding process, Updating with information from a reference frame in a neighboring region. 14. The output video signal sequence according to claim 13, wherein the encoded image includes a macroblock, and further, a specific macroblock is deleted from the encoded video image corresponding to the overlap portion. A method of video encoding, comprising the step of generating. 15. The video coding method according to claim 13, further comprising providing a plurality of area processors, allocating a specific area to each processor, and performing the coding and storing steps. 16. A plurality of region processors for encoding an input signal sequence of a video image, wherein each video image is divided into regions having overlapping portions.
An image selection unit for each region processor to select a specific image region from each of the video images; and a compressed image region having macroblocks according to an encoding process including motion compensation by compressing the selected image region. A compression engine that generates a signal sequence; a local memory that stores a reference frame based on a preceding compressed image region for use in motion compensation of an encoding process; and a compressed image region corresponding to an overlap portion. A macroblock removing device for removing a specific macroblock; a signal sequence linking unit for linking the compressed image sequence signal sequence with a signal sequence from each region processor to generate an output video signal sequence; A reference frame processor that updates each reference frame with information from the reference frame of Video encoder. 17. The video encoder according to claim 16, wherein the encoding process comprises MPEG-2 encoding having a main profile at a main level. 18. The video encoder of claim 16, wherein said compression engine is an MPEG-2 main profile at a main level engine. 19. An input signal sequence of a group of video images in which each video image is divided into regions having overlapping portions, and further, for each region, selecting a specific image region from each of the video images; Compressing the selected image region to generate a compressed image region signal sequence having macroblocks according to an encoding process including motion compensation, and a preceding compressed image region to be used for motion compensation in the encoding process. Storing a reference frame based on, removing a specific macroblock from the compressed image region corresponding to the overlap portion, concatenating the signal sequence of the compressed image region with the signal sequence from each region processor, Generating an output video signal sequence and updating each reference frame with information from a reference frame in an adjacent region. 20. The method of video coding according to claim 19, further comprising a plurality of region processors, each of which is assigned to a specific region for performing the steps of selecting, compressing, storing, deleting and concatenating. . 21. A signal sequence of a group of compressed video images is demultiplexed into a signal sequence of a plurality of regions. At this time, each video image is divided into adjacent regions, and the signal sequence of each region is further divided into a specific region. An associating demultiplexer; and a plurality of area decoders, each of which decodes a signal stream of a specific area into a signal stream of a decoded area according to a decoding process including motion compensation. A reference frame memory for storing reference frames associated with each area decoder, wherein each area decoder takes out a reference frame of an adjacent area and is set to be used for motion compensation in a decoding process; And a multiplexer for multiplexing the signal sequence in the decoded area into a decoded output signal sequence. 22. The decoder according to claim 21, wherein said decoding process for each area decoder comprises MPEG-2 decoding having a main profile at a main level. 23. The decoder according to claim 21, wherein the compressed signal sequence is an ATSC-compliant digital video signal sequence. 24. The decoder according to claim 21, wherein said compressed stream is a digital video signal sequence conforming to an MPEG-2 main profile at a high level. 25. An input signal sequence of a group of compressed video images in which each video image is divided into continuous regions, the compressed signal sequence is demultiplexed into signal sequences of a plurality of regions, and a signal sequence of each region is generated. Is associated with a specific area, the signal sequence of each area is decoded according to the decoding processing including motion compensation, and the reference frame related to each area is stored and referred to the adjacent area to use for the motion compensation of the decoding processing. Regenerating a frame, and further multiplexing the signal sequence of the decoded region into a decoded output signal sequence. 26. The method of video decoding according to claim 25, wherein the step of decoding the signal sequence of each region comprises MPEG-2 decoding having a main profile at a main level. 27. The method of video decoding according to claim 25, wherein said input signal sequence has a digital video signal sequence conforming to the MPEG-2 main profile at a high level.