KR100821922B1 - Local memory controller for mpeg decoder - Google Patents

Abstract

An apparatus for managing a local memory for media decoding is provided to prevent performance deterioration of a memory bus and a process and to prevent excessive access to a system bus by minimizing control of the processor and a use rate of the memory bus by controlling a data flow of a hardwired-logic, and supply a memory path for the data flow. A frame information managing unit(310) manages reference frame and current frame information and performs data exchange with a motion compensation function(240). A macro block line information managing unit(320) manages information about a reference macro block line and a current macro block line, and performs data exchange with an intra prediction function(230). A FIFO(First-In First-Out) managing unit(330) performs data exchange with an IQ/IDCT(Inverse Quantization/Inverse Discrete Cosine Transform) function(220,225).

Description

Local memory management unit for media decoding {LOCAL MEMORY CONTROLLER FOR MPEG DECODER}

1 illustrates an exemplary configuration of a processor employing a conventional multimedia dedicated accelerator.

2 is an exemplary block diagram of the hardwired-logic type multimedia accelerator block of FIG.

3 is an exemplary block diagram of a conventional H.264 / MPEG-4 decoder and a data flow thereof.

4 is a diagram illustrating the concept of intra prediction in media decoding according to the prior art.

5 is a diagram illustrating data access in a macroblock in the media decoding according to the prior art.

6 is an exemplary block diagram of a hardwired-logic type multimedia accelerator block applying a local memory management apparatus for media decoding according to the present invention.

7 is a diagram illustrating a configuration of a macroblock memory control of a rotation type in a local memory management apparatus for media decoding according to the present invention.

FIG. 8 is a diagram illustrating a data flow according to the macroblock memory control configuration of the rotation method of FIG. 7.

9 illustrates an exemplary form of a memory structure for frame storage in a local memory management apparatus for media decoding according to the present invention.

Fig. 10 is a diagram showing an exemplary configuration of a frame information management unit in the local memory management apparatus for media decoding according to the present invention.

<Description of the symbols for the main parts of the drawings>

110: processor core 120: multimedia accelerator

130: post processor 140: memory control unit

210: VLD & Demux 220: IQ

225: IDCT 230: Intra Prediction

240: MC 250: local memory

300: local memory management device for media decoding

310: frame information management unit 320: macroblock line information management unit

330: FIFO management unit 410: DMA host

420: First multiplexer 430: Second multiplexer

440: Reference MB storage unit 450: Current MB storage unit

460: selection unit 470: MB update unit

The present invention relates to a local memory management apparatus for media decoding, and more particularly, to a memory generated by a processor controlling a driving process of macro functions embedded in a multimedia decoding process of a processor having a conventional hard-wired logic. To control the performance of hard-wired-logic data and minimize the control of the memory bus and control of the processor to prevent excessive system bus access due to bus and processor performance degradation and repetitive loading and storage of data. A local memory management apparatus for media decoding that provides a memory path for flow.

The MPEG standard for compressing and decompressing media, such as moving images, is a compression technique that facilitates transmission and storage of moving images continuously changing over time. Such MPEG is implemented in the MPEG-1 / 2/4 and H.264 standards.

The decoding of the video encoded in the MPEG standard is performed through the MPEG decoder.

The conventional MPEG decoder adopts a form of an MPEG decoder composed of a multimedia dedicated processor or DSP, adopts a hardwired-logic multimedia accelerator driven by a host processor, and an MPEG RISC dedicated to a host processor and a one-chip multimedia. Processor type and the like.

 Multimedia-only processor type is mainly used in high-definition or large screen set-top box, high-performance MPEG playback device. In addition, this type of multimedia processor is suitable for power consumption.

In addition, for example, when continuously upgrading the software through a network, various latest codec functions can be provided, but it is difficult to apply to a portable device that needs to be moved.

In addition, in the case of a host processor and a one-chip multimedia-only MPEG RISC processor, for example, it is widely used in a DiVX playback device. In this case, a dedicated processor that handles the built-in MPEG accelerator functions is only responsible for parsing streams and managing the MPEG accelerator.

However, in the case of the host processor and the one-chip-only MPEG RISC processor type, a memory device is required separately from the main processor, and there is a disadvantage of power consumption due to the use of two processors.

Therefore, the MPEG decoder, which is assumed to be used especially in a mobile environment, is preferably configured to adopt a hardwired-logic multimedia dedicated accelerator driven by a host processor.

1 is a diagram showing an exemplary configuration of a processor employing a conventional multimedia dedicated accelerator.

As shown, a processor employing a conventional multimedia dedicated accelerator includes a processor core 110, a multimedia accelerator 120, a post processor 130, a memory controller 140, and a direct memory connection [ Direct Memory Access, DMA] function is used to transfer data through the system bus.

In other words, in the form of an MPEG decoder using a dedicated multimedia accelerator, an Inverse Quantization / Inverse Discrete Cosine Transform (IQ / IDCT) and a variable length are used for an MPEG acceleration function in a memory structure of an existing processor core 110. Multimedia accelerator 120 having functional blocks such as a coding decoder [Variable Length Coding Decoder (VLD)], a motion compensation [MC], an intra prediction, and a deblock filter (DF). And the post processor 130 in the form of hardwired logic, and provide a direct memory access (Direct Memory Access, DMA) function between the local memory and the system bus for each functional block of the multimedia accelerator 120. It is a form to perform the transfer of data using.

The memory controller 140 controls main memories such as SRAM, ROM, and SDRAM.

In this case, an MPEG decoder in which some hardwired-logic multimedia accelerator blocks are embedded may use a local memory bus separate from the main memory bus to reduce the occupancy of the system bus. This requires continuous monitoring of the local memory bus and the processor cores, so that due to local memory size limitations and consequently periodic memory accesses, the utilization of the memory bandwidth of the main processor as compared to the case of using a separate MPEG decoder There is a disadvantage of lowering.

That is, processors with hard-wired logic, such as mobile multimedia processors, contain local memory therein. However, these local memories are designed to be used for the fixed task of different functional blocks, and thus do not perform integrated memory control.

Therefore, the results output through the intra prediction, motion compensation, and IDCT function blocks are stored in the memory such as SDRAM using the interrupt method or the DMA host function in the processor core, and then retransmitted whenever used in other function blocks.

For example, in the process of data transfer or copying by the processor, the memory bus and the processor core are occupied by input of stream data, input of prediction and compensation data or macroblock data, control of local memory, and control of the memory controller. .

Motion compensation blocks also increase memory access by successive read / write operations to the previous frame and current frame memory.

In this case, storing the entire frame in SRAM-type internal memory reduces access to the SDRAM memory, but increases the memory size and power consumption.

In a realistic way, this problem can be compensated for by using a local memory that can store a part of a frame. However, when the image movement on the screen is fast, many memory areas need to be read back from the SDRAM.

This problem is described in more detail as follows.

FIG. 2 is an exemplary block diagram of the hardwired-logic type multimedia accelerator block of FIG. 1.

As shown, incoming stream data is parsed in a variable length coding decoder and demultiplexing (VLD & Demux) functional block 210.

In this case, the macroblock data (MB Data) is transmitted to the IQ / IDCT function blocks 220 and 225, and the prediction and compensation data are the intra prediction blocks 230 and the motion compensation (MC). ) Are each used in function block 240.

The operation results of the IQ / IDCT functional blocks 220 and 225 are stored in the first local memory 250a, and are stored in the main memory such as the SDRAM through the memory control unit 140 for each predetermined amount of calculation.

In the intra prediction function block 230, the operation result of the macroblock data of the previous frame and the macroblock data of the current frame is stored in the second local memory 250b, and the operation result is stored in the memory. It is stored in the main memory such as SDRAM through the control unit 140.

The motion compensation function block 240 generates a current frame through a similar process, stores the current frame in the third local memory 250c, and then stores the current frame in the main memory such as the SDRAM through the memory controller 140.

3 is a diagram illustrating an exemplary block diagram of a conventional media decoder such as H.264 / MPEG-4 and a data flow thereof.

First, the compressed bit stream is entropy decoded to extract coefficients, motion vectors, and headers of each macroblock.

Thereafter, run-level coding and reordering are performed to quantize and generate a transformed macroblock X.

The transformed macroblock X is then inversely quantized (Q ⁻¹ ) and inversely transformed (T ⁻¹ ) to produce a decoded error macroblock D ′ _n .

The decoded motion vector is then used to locate the 16x16 region within the previous frame F ' _n-1 that the decoder has. This region becomes a motion compensated prediction region P.

Then, the decoded error macroblock D ' _n is added to the motion compensated prediction region P to generate a reconstructed macroblock uF' _n .

The reconstructed macroblock uF ' _n is stored to generate a frame F'n which is decoded via a filter.

Also, the reconstructed macroblock uF ' _n may be used in intra prediction.

For example, by sequentially processing the input bitstream in the coefficient of the macroblock, the motion vector, the header extraction, the generation of the transformed macroblock (X), and the generation of the decoded error macroblock (D' _n ). The data to be used in the block is output, but in the motion compensation process using the motion vector, the AC and DC coefficients, and the deblocking filtering process, the reference frame stored in memory, that is, the frame F ' _n-1 (reference) ) And the current frame and conversion parameters. In this case, the main processor intervenes to transfer data and set registers.

This reduces the performance of the processor cores and the efficiency of the system bus.

For example, in the decoding process, the I-picture generates the current frame through the DCT-converted intra prediction and IQ / IDCT that are currently input, and access to the macroblocks located in the intra prediction process occurs. .

Since the P-picture has much linkage with the previous frame when decoding the current frame, the P-picture refers to the previous frame most, and excessive memory access occurs to the main memory such as the SDRAM.

Most processors with built-in multimedia acceleration use local memory. However, intra-block and macro-block data transfer for previous frame images occurs during intra prediction and motion compensation, which degrades the performance of the system bus.

A more detailed description of the intra prediction is as follows.

4 is a diagram exemplarily illustrating a concept of intra prediction in media decoding according to the prior art.

Intra prediction is to use a previously configured macroblock in the same frame in determining the DC / AC coefficient value of the current macroblock or subblock.

As shown in FIG. 4, the macroblock X is predicted from the macroblocks A, B, and C. In this case, the coefficient values of each of the macroblock lines MB Line n and MB Line n + 1 are shown in FIG. 1 or FIG. It should be read randomly through the memory control unit 140 shown in Fig. 2, and thus access of the memory bus occurs.

5 is a diagram illustrating data access in a macroblock in the media decoding according to the prior art.

As shown, in order to reconstruct the macro block X, three read operations to the memory such as the SDRAM through the memory control unit 140 shown in FIG. 1 or 2, that is, the macro block A and the macro block B are performed. And a read operation to the macroblock C occurs.

In addition, the macroblock (B) and the reading of the macroblock (C) can be configured in the burst mode (Burst Mode) to improve the efficiency of using the data bus, but access to the macroblock (A) through the additional memory control unit 140 Access to main memory such as SDRAM is required.

Summary of the Invention An object of the present invention is to reduce the performance of a memory bus and a processor and to repeatedly load data, which is caused by the processor's control of a driving process of embedded macro functions in a multimedia decoding process of a processor having a hardwired logic. Local memory for media decoding that controls hardwired-logic data flow and provides a memory path for data flow to minimize memory bus utilization and processor control to prevent excessive system bus access due to storage It is to provide a management device.

In order to achieve the above technical problem, the present invention provides a local memory management apparatus in a media decoder having a motion compensation function, an intra prediction function, an inverse quantization and an inverse discrete cosine transform function, and controlling a data flow through a memory controller. A frame information management unit that manages frames and current frame information to exchange data with the motion compensation function, and a macroblock that manages data of a reference macroblock line and a current macroblock line to perform data exchange with the intra prediction function. And a line information management unit and a FIFO management unit for performing data exchange with the dequantization and inverse discrete cosine transform functions.

In the local memory management apparatus for media decoding according to the present invention, the local memory management apparatus for media decoding may control the integrated data exchange with the memory controller.

In addition, in the local memory management apparatus for media decoding according to the present invention, the frame information management unit includes a current frame information storage space including information in the form of a macroblock index table and a plurality of information including the information in the form of the macroblock index table. It may include the reference frame information storage space of the.

In the apparatus for managing local memory for media decoding according to the present invention, the plurality of reference frame information storage spaces may be different from each other in the number of macroblock index tables according to the number of references. Can be.

In the apparatus for managing local memory for media decoding according to the present invention, the macroblock index table form may include X offset, Y offset, frame offset, valid display, and sort order information on the entire screen.

In the apparatus for managing local memory for media decoding according to the present invention, the current frame information storage space or the reference frame information storage space may include frame number, X offset, Y offset, and window size information.

In the local memory management apparatus for media decoding according to the present invention, the macroblock line information management unit includes: a macroblock update unit for updating macroblock information based on a calculation result of the intra prediction function; A macroblock storage unit for storing information about the reference macroblock and information about the reference macroblock in a rotational manner, a multiplexer which multiplexes an output from the memory controller or an output of the macroblock updater to the macroblock storage unit, and The macroblock storage unit may include a selection unit for selecting from the information on the current macroblock or the information on the reference macroblock to transmit to the intra prediction function.

In the apparatus for managing local memory for media decoding according to the present invention, the macroblock storage unit includes: a current macroblock storage unit storing information on the current macroblock; and a reference macro storing information on the reference macroblock. It may include a block storage unit.

In the local memory management apparatus for media decoding according to the present invention, the multiplexer may include: a first multiplexer which multiplexes an output from the memory controller or an output of the macroblock updater to the reference macroblock storage; And a second multiplexer for multiplexing an output from a memory controller or an output of the macroblock updater to the current macroblock storage.

In the apparatus for managing local memory for media decoding according to the present invention, the selector may select from an output of the current macroblock storage unit or the reference macroblock storage unit to output the intra prediction function.

In the apparatus for managing local memory for media decoding according to the present invention, the macroblock line information management unit may further include a DMA host for mediating memory access between the memory control unit and the multiplexer.

Hereinafter, an embodiment of a local memory management apparatus for media decoding according to the present invention will be described in more detail with reference to the accompanying drawings.

6 is an exemplary block diagram of a hardwired-logic type multimedia accelerator block to which a local memory management apparatus for media decoding according to the present invention is applied.

As shown, in the conventional hardwired-logic type of multimedia accelerator block, a local memory management apparatus 300 for media decoding according to the present invention is included.

The local memory management apparatus 300 for media decoding according to the present invention is designed to use conventional local memories for fixed tasks of different functional blocks, thereby improving the disadvantage of failing to perform integrated memory control.

As illustrated, the local memory management apparatus 300 for media decoding according to the present invention includes a frame information management unit 310, a macroblock line information management unit 320, and a FIFO management unit 330.

As illustrated, the frame information manager 310 manages a reference frame and current frame information to exchange data with the motion compensation function MC 240.

As illustrated, the macroblock line information manager 320 manages information of the reference macroblock line and the current macroblock line to perform data exchange with the intra prediction function 230.

The FIFO management unit 330 manages macroblock information to perform data exchange with inverse quantization and inverse discrete cosine transform functions (IQ / IDCT, 220, 225).

The local memory management apparatus 300 for media decoding according to the present invention includes information about a reference frame such as a previous frame, a reference macroblock line such as a current frame, a previous macroblock line, and a current macroblock line. Local memory control is possible.

In addition, the local memory management apparatus 300 for media decoding according to the present invention may control the integrated data exchange with the memory controller 140.

 That is, access to the memory controller 140 may be minimized by integrally performing access to the memory controller 140 separately performed by each of the conventional local memory devices.

Hereinafter, a detailed configuration of a local memory management apparatus for media decoding according to the present invention will be described.

FIG. 7 is a diagram illustrating a configuration of a macroblock memory control of a rotation type in a local memory management apparatus for media decoding according to the present invention.

In the data access to the macroblock for intra prediction of the media decoding according to the prior art with reference to FIG. 5, the memory control unit 140 shown in FIG. 1 or 2 is used to reconstruct the macroblock X. Three read operations to a memory such as an SDRAM occur. Alternatively, the burst mode may be used to improve the efficiency of the data bus, but access to the macroblock A requires access to main memory such as SDRAM through the additional memory controller 140. It was described.

In order to reduce access to main memory such as SDRAM through the memory control unit 140 in the intra prediction process, the macroblock line information management unit 320 is rotated in the local memory management apparatus for media decoding according to the present invention. It can be implemented in the form of a macroblock memory control configuration.

As shown, the macroblock memory control configuration of the rotation scheme may be provided in particular in connection with data exchange between the memory control unit 140 and the intra prediction function 230.

As shown, in the local memory management apparatus for media decoding according to the present invention, a rotational macroblock memory control configuration includes a DMA host 410, a first multiplexer 420, a second multiplexer 430, A reference macroblock storage unit 440, a current macroblock storage unit 450, a selection unit 460, and a macroblock updater 470 are included.

The DMA host 410 mediates memory access between the memory controller 140, the first multiplexer 420, and the second multiplexer 430.

The first multiplexer 420 and the second multiplexer 430 selectively store the data according to intra prediction under the control of the macroblock updater 470, and store the current macroblock. Stored in each section 450.

The reference macroblock storage unit 440 and the current macroblock storage unit 450 store information on the reference macroblock and the current macroblock, respectively, in a rotating manner.

The selector 460 selects the operation data of the reference macroblock and the current macroblock from the intra-prediction macroblock storage unit 440 and the current macroblock storage unit 450 to transmit to the intra prediction function 230.

The macroblock updating unit 470 may include a reference macroblock storage unit 440 from the intra prediction function 230, a first multiplexer 420, and a second unit to store data in the current macroblock storage unit 450. The operation of the multiplexer 430 is controlled.

That is, the macroblock updater 470 controls to update information on the reference macroblock and the current macroblock based on the operation result of the intra prediction function 230.

As described above, if a local memory for storing two macroblocks of the reference macroblock storage unit 440 and the current macroblock storage unit 450 is stored, and reference is performed for each macroblock line by line, the previous macroblock line The frequency of memory accesses to may decrease. In addition, access to main memory, such as SDRAM, required for writing current macroblock lines can also be reduced.

That is, compared to the data access method of the macroblock of FIG. 5, the macroblock memory control configuration of the rotation method of FIG. 7 provides reference macroblock storage for information on macroblocks B and C necessary for generating the macroblock X. FIG. The macroblock A is stored in the current macroblock storage unit 450 through previous intra prediction.

FIG. 8 is a diagram illustrating a data flow according to the macroblock memory control configuration of the rotation method of FIG. 7.

As shown, the processing at each step in the rotating macroblock memory control arrangement according to the invention is shown.

When performing an operation on the n + 1th macroblock line, the data of the nth current macroblock information is received as it is and used as the n + 1th reference macroblock information, and the current on the n + 1th macroblock line. The macroblock information is allocated to the nth reference macroblock information portion. As such, the same memory area stores current macroblock information or reference macroblock information step by step, and this method is referred to as a " rotational macroblock memory control configuration " in the present invention.

In the case of inter prediction, it is as follows.

In the inter prediction process, a memory block may be configured as part of a macroblock line to reduce access to main memory such as SDRAM, and the frame memory may be controlled through an X offset, a Y offset, and a frame offset using an index.

In other words, the memory access for reading the reference frame picture required for the motion compensation process can be reduced in this manner. In addition, by replacing the index register for storing such information, it is possible to reduce the copy operation required for data conversion between frames.

9 is a diagram illustrating an exemplary form of a memory structure for storing frames in the local memory management apparatus for media decoding according to the present invention.

As shown in FIG. 9, a memory structure for storing a frame includes a structure for storing a current frame and a storing of a reference frame.

Each macroblock is composed of, for example, 16x16 pixels, and one macroblock index can store a screen of 128x16 pixels.

For example, as shown for the macroblock index MB Index 0, storage is performed for eight macroblocks.

In addition, the structure of each macroblock index, as shown, the number of X, Y offset (X Offset, Y Offset) and the frame offset (Frame Offset) on the whole screen, the field (In Use), the macro for valid display It includes a field (Arrange Order) indicating the sort order between each macroblock of a block line.

10 is a diagram showing an exemplary configuration of a frame information management unit in the local memory management apparatus for media decoding according to the present invention.

As shown, only necessary parts can be loaded and used for a reference frame used for motion compensation, and only necessary parts can be updated individually.

That is, the frame information management unit of the local memory management apparatus for media decoding may include three frame storage spaces, such as a storage space for a current frame and a storage space for two reference frames.

In this case, each frame storage space includes a frame number, an X offset offset, a Y offset offset, window size information, and a plurality of macroblock index tables.

The space for storing the current frame and the storage space for the previous frame (the reference frame (1)) where the most references occur are composed of a number of macroblock index tables (MB Index Table) as shown, and before one frame. The storage space for the reference frame (reference frame 2) of is composed of two macroblock index tables as shown, for example, because the number of references is relatively small.

Although the configuration of the present invention has been described in detail, these are merely illustrative of the present invention, and various modifications may be made by those skilled in the art without departing from the essential characteristics of the present invention. This will be possible. Therefore, the embodiments disclosed herein are not intended to limit the present invention but to describe the present invention, and the spirit and scope of the present invention are not limited by these embodiments. If the scope of the present invention is to be interpreted by the claims below, it should be interpreted that all techniques falling within the scope of the present invention are included in the scope of the invention.

As described above, according to the present invention, performance degradation and repetitive performance of the memory bus and the processor that occur due to the processor's control of the driving process of the built-in macro functions in the multimedia decoding process of the processor incorporating the hardwired logic are conventional. In order to prevent excessive system bus access due to data loading / storing process, hardwired-logic data flow is controlled and memory path for data flow is minimized to minimize memory bus utilization and processor control. Improve the performance of your mobile multimedia processor.

Claims (11)

A local memory management apparatus in a media decoder having a motion compensation function, an intra prediction function, an inverse quantization and an inverse discrete cosine transform function, and controlling a data flow through a memory controller, A frame information manager which manages a reference frame and current frame information to exchange data with the motion compensation function; A macroblock line information management unit for managing data of a reference macroblock line and a current macroblock line and performing data exchange with the intra prediction function; FIFO management unit performing data exchange with the inverse quantization and inverse discrete cosine transform function Local memory management apparatus for media decoding comprising a. delete The method of claim 1, wherein the frame information management unit, A current frame information storage space including macroblock index table type information; And a plurality of reference frame information storage spaces containing information in the form of the macroblock index table. The method of claim 3, And wherein the plurality of reference frame information storage spaces are different in number for each of the plurality of reference frame information storage spaces according to the number of references. The method of claim 3, The macroblock index table form is And a X offset, Y offset, frame offset, valid indication, and sort order information on the entire screen. The method of claim 3, And the current frame information storage space or the reference frame information storage space includes frame number, X offset, Y offset, and window size information. The method of claim 1, wherein the macroblock line information management unit, A macroblock updater for updating macroblock information based on a result of the operation of the intra prediction function; A macroblock storage unit which stores information on the current macroblock and information on the reference macroblock; A multiplexer for multiplexing an output from the memory controller or an output of the macroblock updater to the macroblock storage; A selector which selects from the information on the current macroblock or the information on the reference macroblock and transmits the information to the intra prediction function Local memory management apparatus for media decoding comprising a. The method of claim 7, wherein The macroblock storage unit, A current macroblock storage unit for storing information on the current macroblock; Reference macroblock storage unit for storing information about the reference macroblock Local memory management apparatus for media decoding comprising a. The method of claim 8, wherein the multiplexing unit, Under the control of the macroblock updater, A first multiplexer which multiplexes an output from the memory controller or an output of the macroblock updater, and outputs the multiplexer to the reference macroblock storage; A second multiplexer which multiplexes an output from the memory controller or an output of the macroblock updater to the current macroblock storage; Local memory management apparatus for media decoding comprising a. The apparatus of claim 8, wherein the selector selects an output from the current macroblock storage unit or the reference macroblock storage unit to output to the intra prediction function. 10. The method of claim 7, wherein the macroblock line information management unit, DMA host for mediating memory access between the memory controller and the multiplexer Local memory management apparatus for media decoding further comprising.

Images