KR20090010860A - Method and apparatus for decoding multimedia based on multicore processor - Google Patents

Abstract

A multimedia decoding method and an apparatus thereof are provided to effectively use resources a multiprocessor through a hybrid mode, thereby efficiently processing multimedia data having much capacity and operation. A queue for least one or more operation works is generated in a central processor with respect to inputted multimedia data(910). The central processor and an arithmetic processor perform the operation work. At least one or more partial arithmetic processor of the arithmetic processors performs a motion compensation task according to a data area in which the multimedia data is divided(920). One arithmetic processor of the arithmetic processors performs a de-blocking task for the multimedia data(940).

Description

Method and apparatus for decoding multimedia based on multicore platform TECHNICAL FIELD

The present invention relates to a method and apparatus for decoding multimedia data, and more particularly, to an efficient design method and an efficient decoding apparatus for a multi-core platform-based multimedia decoding system.

Due to the large capacity of the H.264 format and a high amount of computation, an efficient decoding method of the H.264 format has been proposed in an asymmetric multicore platform environment. However, unlike MPEG-2, which can process slice-by-slice, H.264 has not only data interdependency but also interdependence of data within one screen, so that a parallel processing scheme of a decoding system based on a multicore platform is implemented. It's hard to be.

Conventional partitioning schemes include a data partitioning scheme for partitioning data to be processed by a processor and a functional partitioning scheme for dividing a processing module step like a pipelined scheme.

1 illustrates an example of a multicore platform system with a functional partitioning scheme.

The functional partitioning scheme includes a plurality of processors 110, 120, 130, and 140 so that specific functions are assigned to each processor. The processor 1 110 illustrated in FIG. 1 is assigned a data read function 112, a preprocessing and initialization function 114, and a data storage function 116. The processor 2 120 is assigned an entropy decoding function 122. do. Process 3 130 is assigned an inverse transform and inverse quantization function 132 and an intra prediction and motion compensation function 134. The deblocking function 142 is assigned to the processor 4 140.

In the functional partitioning scheme, it is difficult to guarantee a constant performance when the workload of each distributed computing module is not constant. That is, since the processing time 150 of the processor 1, the processing time 160 of the processor 2, the processing time 170 of the processor 3, and the processing time 180 of the processor 4 are all different, the processing time of the multiprocessor is the longest. Because of the processing time 170 of the processor 3, which is a job processing time, a critical path equal to the excess processing time 190 is generated. Thus, parallelism and utilization of multicore systems are reduced.

2A and 2B show an example of a multicore platform system with a data partitioning scheme.

Referring to FIG. 2A, one example of a data partitioning method is to divide one frame 200 into several slices 210, 212, and 214, and to decode the slices in different processors 220, 230, and 240 for each slice. To perform. That is, all the decoding processes of one processor are performed. The data partitioning scheme guarantees high parallelism for simple data processing. However, if there are interdependencies between the data, the data partitioning scheme is difficult to implement and additional work is required to resolve the dependencies, resulting in a drastic drop in performance. Therefore, it is not suitable for H.264-based decoding system that needs to perform not only the inter prediction but also the intra prediction.

In addition, referring to FIG. 2B, it is difficult to predict the relationship between the data size and the computational burden of the slices 260, 270, and 280 in which the frame 250 is divided. That is, the processor 1 220 and the processor 3 240 that process the slice 1 260 and the slice 3 280, respectively, have a small computational burden, but the processor 2 230 that processes the slice 2 270 has a data size. ) Only increases the computational burden. In other words, due to the disparity of the divided data size, the computational load between processors is unevenly allocated, thereby reducing the efficiency of resource utilization.

In addition, the existence of dependencies between each data can complicate the implementation of parallel processing architectures and delay processing. In addition, while the resources of the partial cores constituting the multicore platform are limited, it is inefficient because each core must have data for the entire calculation process.

3A and 3B show differences in the distribution of data and instructions according to application characteristics.

Both data and instructions are required to handle computational tasks. As shown in FIG. 3A, when the size of the data 310 stored in the memory 300 is small while the size of the command 320 is large, a functional partitioning method is advantageous. On the contrary, as shown in FIG. 3B, the data 330 is advantageous when the size of the data 330 stored in the memory 300 is large while the size of the instruction 340 is small.

However, the characteristics of data and instructions may vary according to application characteristics, and the characteristics of data and instructions of each module constituting a program may also vary. Therefore, if the parallel processing partitioning scheme of the multiprocessor is fixed to either a functional partitioning scheme or a data partitioning scheme, it cannot flexibly correspond to the characteristics of data and instructions.

In addition, the size of the instructions of the H.264 decoder based on a single core system is 820 kilobytes and the data size is 200 kilobytes, whereas the local memory is only 256 kilobytes. Therefore, it is difficult to efficiently implement an H.264 decoding system with limited resources of local memory.

Disclosure of Invention Problems to be Solved by the Invention An object of the present invention is to effectively process parallel data using multiprocessor resources in order to efficiently process high-capacity and computation-intensive multimedia data in a multi-core platform-based multimedia decoding method and apparatus.

For efficient parallel processing of multiprocessors, a method of using the advantages of the data partitioning method and the functional partitioning method and distributing the operation load evenly for each processor is proposed. In addition, a queuing method for efficient computational work instructions of multiprocessors and an efficient data communication method between the multiprocessors are proposed.

In order to achieve the above object, the multi-core platform-based multimedia decoding method according to an embodiment of the present invention, in the multi-core platform-based multimedia decoding method comprising a central processor and a plurality of arithmetic processor, input Generating a queue of at least one computational task for the centralized processor and the arithmetic processor to perform, for the multimedia data that has been processed; Performing a motion compensation operation for each data region in which the multimedia data is divided in at least one or more of the computing processors; And performing, by one of the arithmetic processors, a deblocking operation on the multimedia data.

The multimedia decoding method according to an embodiment may include: reading the input multimedia data by the central processor; Performing an initialization operation for decoding on the read multimedia data in the central processor; Entropy decoding the initialized multimedia data in the central processor; Performing at least one of an inverse frequency transform operation, an inverse quantization operation, and an intra prediction operation on the multimedia data in the central processor; And storing, by the central processor, the multimedia data on which the deblocking operation has been performed, wherein the queue generating step generates a queue for arithmetic operations on the entropy decoded multimedia data.

The queue creation step of an embodiment includes putting a work parameter block containing work parameters for the arithmetic tasks into a queue for each arithmetic task.

The queue generating step of the embodiment may further include transmitting the work parameter block from a queue for the arithmetic work to an arithmetic processor corresponding to each arithmetic work queue.

In an embodiment, the step of transmitting a work parameter block may include: determining a work parameter block that may be processed by the operation processor for a predetermined unit time; And transmitting the determined job parameter block to the operation processor at each predetermined unit time.

The multimedia decoding method according to an embodiment of the present invention comprises the steps of: dividing the multimedia data into at least one data area; And determining a separate operation processor to perform a motion compensation task on each of the divided data regions, wherein the queue generating step includes: when generating a queue for the motion compensation task, performing the motion compensation task; Each of the arithmetic processors to be executed generates a queue including a block of work parameters to be performed on the data area.

In an exemplary embodiment, the data area partitioning step may include calculating an operation burden of an operation processor for a motion compensation operation on the multimedia data; Determining the number of divisions of the multimedia data into a data area in consideration of an operation burden of the operation processor; And dividing the multimedia data into the data area by the number of divisions, wherein the operation processor determining step determines the number of operation processors equal to the number of divisions of the multimedia data.

In an exemplary embodiment, the data area segmentation may include detecting data having dependencies on the multimedia data; And determining the divided data area so that the data which are dependent on each other are not separated.

The determining of the data area of the present embodiment may include: when the first data of the detected data is dependent on two or more other data, one of the other data having the dependency as data to be included in the same data area as the first data. Determining the data.

The queue generating step may include storing a job parameter block for each divided data area in a queue for the motion compensation job according to the processing order of each data area, and transmitting the job parameter block. The step includes transmitting, from the queue for the motion compensation task, a task parameter block for each of the divided data regions according to the processing order of the respective data regions.

In the multi-core platform-based multimedia decoding method according to an embodiment, the synchronization signal between the central processor and the computing processor is input and output through a mailbox method.

In the multi-core platform-based multimedia decoding method according to an embodiment, data between the memory of the central processor and the memory of each of the computing processors is input and output through a direct memory access (DMA) method.

The multi-core platform-based multimedia decoding method according to an embodiment further includes synchronizing motion compensation tasks for the respective data regions in the operation processor for the motion compensation task.

The multi-core platform-based multimedia decoding method according to an embodiment of the present invention further comprises the step of processing the respective operations of the central processor and the computing processors in a pipeline structure in parallel.

In the parallel processing of the embodiment, the computational tasks of the central processor and the computational processors may be defined as: n is greater than 1 and less than N when the multimedia data is divided into N data units per data processing interval. If it is an integer, performing an entropy decoding operation on the (n-1) th data unit in the central processor and generating the queue; The operation processor for the motion compensation task performs a motion compensation task on the (n-1) th data unit, and the central processor performs an entropy decoding operation on the (n) th data unit and performs the queue. Generating; And performing an inverse frequency transform operation, an inverse quantization operation, and an in-prediction operation on the (n-1) th data unit in the central processor, and the (n) th data unit in the arithmetic processor for the motion compensation task. Performs a motion compensation operation on the processor, performs an entropy decoding operation on the (n + 1) th data unit in the central processor, generates the queue, and executes the (n−) operation processor in the operation processor for the deblocking operation. 1) restoring the (n-1) th data unit by performing a deblocking operation on the first data unit.

In order to achieve the object to be solved, the multi-core platform-based multimedia decoding apparatus according to an embodiment of the present invention, at least for the central processor and the operation processor to perform the input multimedia data in the central processor; A queue generator for generating a queue for one or more arithmetic operations; A motion compensation task performing unit configured to perform a motion compensation task for each data region in which the multimedia data is divided in at least one or more of the computing processors; And a deblocking task execution unit configured to perform a deblocking operation on the multimedia data by one of the arithmetic processors.

The multimedia decoding apparatus according to an embodiment of the present disclosure, in the central processor, reads the input multimedia data, performs an initialization operation for decoding the read multimedia data, and entropys the initialized multimedia data. And decode, perform at least one of an inverse frequency transform operation, an inverse quantization operation, and an intra prediction operation on the multimedia data, and store the multimedia data on which the deblocking operation is performed. Create a queue for arithmetic operations on entropy-decoded multimedia data.

The present invention also includes a computer-readable recording medium having recorded thereon a program for implementing a multi-core platform-based multimedia decoding method according to an embodiment of the present invention.

The multi-core platform-based multimedia decoding method and apparatus of the present invention efficiently utilizes multiprocessor resources by using a hybrid method that combines the advantages of the data partitioning method and the functional partitioning method. Can be processed.

In addition, by calculating the computational burden for each divided data region, the computational burden among the computing processors is equally distributed, thereby improving the parallelism of the multiprocessor.

By sequentially queuing the job parameters according to the processing order of the processors in advance in the central processor, the processors can perform efficient computational work of the pipeline structure. In addition, the parallel processing of multiprocessors is facilitated by ensuring the synchronization of tasks between multiprocessors.

For convenience of description of the embodiments of the present invention, the data unit used herein is defined in advance.

The data processing unit refers to a unit of a frame, slice, picture, etc. that divides the multimedia data by a predetermined criterion.

The data unit refers to a portion in which one data processing unit is divided into several parts so that one processor may sequentially process the data processing unit having a large amount of data more efficiently. For example, a frame may be divided into a plurality of data units.

The data area refers to an area including at least one data processing unit so that multiple processors can simultaneously process the multimedia data for efficient decoding and decoding. For example, the data area is an area including at least one frame.

Hereinafter, referring to FIGS. 4 to 9, a method of configuring and operating a multimedia decoding method and a multimedia decoding apparatus based on a multicore platform according to an embodiment of the present invention will be described in detail.

4 is a block diagram of a multi-core platform-based multimedia decoding apparatus according to an embodiment of the present invention.

The multi-core platform-based multimedia decoding apparatus 400 according to an embodiment of the present invention includes a queue generator 410, a motion compensation task performer 420, an inverse frequency transform and dequantizer 430, and a deblocking task. It includes an execution unit 440. Since the multimedia decoding apparatus 400 is based on a multicore platform, the multimedia decoding apparatus 400 includes a plurality of arithmetic processors, and an embodiment includes a central processor and at least one arithmetic processor.

The queue generator 410 generates a queue for at least one or more computational tasks to be performed by the central processor and the computational processor on the input multimedia data. The creation of the queue is done by the central processor. Thus, one embodiment of the queue generator 410 is an element of the central processor.

According to an embodiment of the multimedia decoding apparatus 400, before the queue is generated by the queue generator 410 of the central processor, the central processor reads the input multimedia data and initializes the decrypted multimedia data. Do this. The central processor also performs entropy decoding, inverse frequency transform, inverse quantization, and intra prediction on the initialized multimedia data. An embodiment of the queue generator 410 generates a queue for arithmetic operations on entropy-decoded multimedia data.

An embodiment of the queue generator 410 determines a work parameter block that the arithmetic processor can process for a predetermined unit time to transmit a work parameter block, and transmits the determined work parameter block to the arithmetic processor every predetermined unit time. do.

An embodiment of the queue generator 410 inserts a work parameter block including work parameters for arithmetic tasks into a queue for each arithmetic task. In addition, an embodiment of the queue generator 410 transmits a work parameter block to a calculation processor corresponding to each arithmetic work queue from a queue for the arithmetic work.

Although not shown in the drawing, the multimedia decoding apparatus 400 according to an embodiment includes a data area divider for dividing the multimedia data into at least one data area. The multimedia decoding apparatus 400 according to an embodiment determines a separate operation processor for performing a motion compensation operation on each data region divided by the data region dividing unit.

According to an embodiment of the queue generator 410, when generating a queue for a motion compensation task, each task parameter block to be performed on a corresponding data area in each of the arithmetic processors to perform a motion compensation task is performed. Create a queue containing.

In one embodiment, the data area partitioner calculates an operation burden of an operation processor for a motion compensation operation on the multimedia data to divide each data processing section of the multimedia data into at least one data area, and calculates an operation burden of the operation processor. In consideration of the amount, the number of divisions of the multimedia data into the data area is determined, and the multimedia data is divided into the data areas by the number of divisions. In addition, as the operation processor for the motion compensation task, the same number of operation processors as the number of divisions of the multimedia data are determined.

An embodiment of the data area partitioner detects data that is dependent on each other among the multimedia data, and determines a data area to be partitioned so that the data that is dependent on each other is not separated.

Further, in one embodiment of the data area partitioning unit, in order to partition the data area so that data that is dependent on each other is not separated, when the first data among the data detected as being dependent on each other depends on two or more other data, One data among other data having dependency as the data to be included in the same data area as the first data is determined.

An embodiment of the queue generator 410 stores a job parameter block for each divided data area in a queue for a motion compensation job according to a processing order of each data area, and stores the job parameter block from the queue for a motion compensation job. The work parameter block for each divided data area is transferred according to the processing order of each data area.

The motion compensation task execution unit 420 performs a motion compensation task for each data region in which multimedia data is divided in at least one or more of the arithmetic processors.

The reverse frequency converted light dequantization unit 430 performs an inverse frequency transform operation and an inverse quantization operation on data on which the motion compensation operation is performed. In one embodiment, the inverse frequency transform and inverse quantizer 430 is part of a central processor.

The deblocking task execution unit 440 performs a deblocking task on the multimedia data on which one of the arithmetic processors has been subjected to the inverse frequency conversion and inverse quantization.

In an embodiment of the multimedia decoding apparatus 400, the central processor stores the multimedia data on which the deblocking operation is performed.

5 shows a distribution of calculation burdens for each calculation operation.

The graph 500 illustrated in FIG. 5 illustrates an example of time taken for decoding multimedia data in the multimedia decoding apparatus 400 as a ratio for each operation.

15% of the total decoding time in the main loop 510 for multimedia data, 3% of the total time in the initialization operation 520, 5% of the total time for the entropy decoding operation 530, motion compensation Task 540 is 52% of total time spent, intra prediction job 550 is 1% of total time spent, inverse quantization and reverse frequency conversion job 560 is 3% of total time spent, deblocking job 570 ) Takes 21% of the total time. According to FIG. 5, in various computational tasks of the multimedia decoding task, the time required for each computational task is different, and accordingly, computational burden of processors performing the computational task is also different.

Accordingly, in one embodiment of the present invention, the central processor performs a main loop operation 510, an initialization operation 520, an intra prediction operation 550, an inverse quantization operation and an inverse frequency conversion operation 560, and a plurality of operations. The motion compensation task 540 is performed in the two computing processors, and the deblocking operation 570 is performed in another computing processor.

6 illustrates a hybrid partitioning scheme of a multimedia decoding apparatus according to an embodiment of the present invention.

The multimedia decoding apparatus 400 is based on the multicore platform 600 and includes a central processor 610 and a plurality of arithmetic processors 620. The arithmetic processors 620 of one embodiment include three arithmetic processors 630, 640, and 650 for motion compensation tasks and arithmetic processor 660 for deblocking tasks.

Multimedia decoding apparatus 400 of the present invention is a central processor 610 in the data read operation 611, pre-processing and initialization operation 612, entropy decoding operation 613 and queue creation operation 614 for the operation parameters, Perform inverse frequency transform operation, inverse quantization operation and intra prediction operation 616, data storage operation 618, and the motion compensation operation 635, 645, 655 in a set of computing processors 630, 640, 650. The deblocking operation 667 is performed by another operation processor 660. This corresponds to a functional partitioning scheme in a multicore platform.

In addition, the multimedia decoding apparatus 400 of the present invention divides the data processing unit of the multimedia data into at least one or more data areas, and performs motion compensation operations 635, 645, and 655 for each data area as a separate operation processor ( 630, 640, 650). This corresponds to the data partitioning method in the multicore platform.

Accordingly, the multimedia decoding apparatus 400 of the present invention combines a functional partitioning method and a data partitioning method in order to share the amount of calculation for each processor in consideration of the computational burden of the computational work of each multimedia decoding operation. Adopt.

In order to determine the number of arithmetic processors for a motion compensation task, an embodiment of the present invention calculates a computational burden of the arithmetic processor for a motion compensation task for multimedia data. In consideration of the calculated computational burden of the computing processor, the number of divisions of the multimedia data into the data region is determined, and the multimedia data is divided into the data regions by the determined number of divisions. Thus, the same number of computation processors as the number of divisions of the determined multimedia data is determined.

In one embodiment of the invention, signaling for synchronization of computational work between the central processor 610 and other computational processors 630, 640, 650, 660 is via a mailbox.

7 illustrates a queuing method and a communication method between processors according to an embodiment of the present invention.

An embodiment of the queue generator 410 related to the central processor 610 generates a queue including work parameters related to arithmetic work to be performed by the arithmetic processors. That is, the embodiment of the queue generator 410 generates a queue by sequentially stacking a parameter block including work parameters related to a calculation task and related data in a queue according to the order of the calculation tasks processed by the calculation processor. do. Accordingly, one embodiment of the queue generator 410 generates a queue for a motion compensation task, a queue for a deblocking task, and a respective queue for an inverse frequency transform task / inverse quantization task / intra prediction task.

An example is given as the motion compensation operation shown in FIG. The central processor 610 queues the work parameter blocks 730, 740, and 750 for performing the motion compensation task in the processors 630, 640, and 650 for the motion compensation task. Stacked sequentially at 710. Main memory represents the memory of the central processor.

In addition, each of the job parameter blocks 730, 740, and 750 corresponds to the arithmetic processors 630, 640, and 650 for the motion compensation task for each divided data region. , 640, 650). In one embodiment, the data communication between the main memory and the memories of the computing processors 630, 640, 650 is through a direct memory access (DMA) scheme, so that the computing processor from the queue 710 for motion compensation work. Data output to 630, 640, and 650 is based on the DMA method.

As described above, signaling for synchronization of computational tasks between the central processor 610 and the computational processors 630, 640, 650 is via a mailbox.

Since the operations between the processors of the present invention are performed in parallel in a pipelined structure, the central processor 610 may predetermine a job parameter block for a data unit for which motion compensation is to be performed in the current computing processors 630, 640, and 650. Stacked in the queue 710 of the memory. Parallel processing of the pipeline structure of the multimedia decoding apparatus 400 according to an embodiment will be described later in detail with reference to FIG. 8.

The data area of one embodiment may be divided into macroblock units within a frame. One data area may include a plurality of macro blocks, and one operation processor performs motion compensation on one data area during a current data processing period.

An embodiment of the data area partitioner may detect data that are dependent on each other among multimedia data and prevent the data areas that are dependent on each other from being separated so that data areas that are dependent on each other are not divided. Decide For example, the data area is determined so that P frames and B frames that are dependent on each other are included in the same data area.

According to an embodiment of the present invention, when the first data of the detected data is dependent on two or more other data, the data area partitioner determines one of the other data depending on the data to be included in the same data area as the first data. do.

For example, if a B frame (first data) refers to a previous P frame and a next P frame (two or more other data that depend on the first data), the boundary between the previous P frame and the B frame, or the B frame and the next When the data area is to be divided at the boundary between the P frames, the data area dividing unit determines whether to divide the data area at any one of the two boundaries.

8 illustrates a processor parallel processing scheme of a pipeline structure according to an embodiment of the present invention.

8 illustrates a frame as an example of a data processing unit, and a quadrant portion of the frame as an example of a data unit that is a partial unit of the data processing unit.

Since the exemplary embodiment of the multimedia decoding apparatus 400 depends on the processor parallel processing scheme of the pipeline structure, the multimedia decoding apparatus 400 uses the following procedure.

1. After the central processor 810 performs an entropy decoding operation on the first data unit and generates a queue 811, the central processor 810 outputs the first data unit to an operation processor for a motion compensation task.

2. While the arithmetic processors 820, 830, and 840 for motion compensation operations perform motion compensation operations on the first data unit (821, 831, 841), the central processor 810 for the second data unit. Entropy decryption is performed and a queue is created (813).

During this data unit execution period, the first data unit is output to the central processor 810, and the second data unit is output to the operation processors 820, 830, and 840 for the movement task.

 When the arithmetic operations 821, 831, and 841 of the arithmetic processor for the motion compensation task and the arithmetic operations 813 of the central processor are completed, the central processor 810 and the processors 820, 830, and 840 for the motion compensation task are completed. You are ready to perform the operation operation on the next data unit.

3. While the arithmetic processors 820, 830, and 840 for the motion compensation task perform the motion compensation tasks on the second data unit (823, 833, 843), the central processor performs inverse frequency transform on the first data unit. A task, an inverse quantization operation, and an intra prediction operation are performed 814, and after the calculation operation of the central processor for the first data unit is completed, the first data unit is output to the processor 850 for the deblocking operation.

The processor 850 for the deblocking operation performs a deblocking operation on the first data unit (851), and finally restores the first data unit 861 on which the deblocking operation is completed.

After the central processor completes an inverse transform operation, an inverse quantization operation, and an intra prediction operation 814 on the first data unit, the central processor performs an entropy decoding operation on the third data unit and generates a queue 815.

When the computational tasks 820, 830, and 840 of the processor for the motion compensation task complete the computational tasks 823, 833, and 843 for the second data unit, the second data unit 862 is output to the central processor.

Once all of the arithmetic operation 851 of the arithmetic processor for deblocking operation, arithmetic operations of the arithmetic processor for motion compensation operation 823, 833, 843 and arithmetic operation of the central processor 815 are completed, The next job operation is ready to be performed by the parameter block output to each processor from the queue created for the data unit.

4. The parallel processing of the pipeline structures 1, 2 and 3 described above is repeated. Accordingly, the inverse frequency transform operation, the inverse quantization operation, and the infrastructure prediction operation 816 and the entropy decoding operation and the queue generation operation 817 for the fourth data unit are performed on the second data unit in the central processor 810. The motion compensation operation 825, 835, 845 for the third data unit is performed in the operation processors 820, 830, and 840 for the motion compensation operation, and the operation processor 850 for the deblocking operation is performed for the second time. Deblocking operation 853 is performed on the data unit. In addition, when the deblocking operation 853 is completed, the second data unit is restored.

One embodiment of the multimedia decoding apparatus 400 ensures synchronization of computational tasks between the central processor and the computational processor. In addition, according to an embodiment of the arithmetic processor for a motion compensation task, a motion compensation task is simultaneously performed by a separate arithmetic processor for each data region, and thus, motion compensation tasks 821, 831, 841/823, 833 for each data unit are performed. , 843/825, 835, 845 are all performed simultaneously on each computing processor.

9 is a flowchart of a multi-core platform-based multimedia decoding method according to an embodiment of the present invention.

In operation 910, for the input multimedia data, a queue is generated for at least one or more computational tasks to be performed by the central processor and the computational processor at the central processor.

According to an embodiment of the multimedia decoding method, an input multimedia data is read by a central processor, an initialization operation for decoding is performed at the central processor on the read multimedia data, and an entropy decoding operation is performed at the central processor on the initialized multimedia data. This is done. In addition, at least one of an inverse frequency transform operation, an inverse quantization operation, and an intra prediction operation is performed on the multimedia data in the central processor, and the multimedia data on which the deblocking operation is performed is stored in the central processor.

In one embodiment, a queue is created for arithmetic operations on entropy decoded multimedia data.

In operation 920, a motion compensation operation is performed for each data area in which multimedia data is divided in at least one or more of the computing processors.

In operation 930, an inverse frequency transform operation and an inverse quantization operation are performed on the data on which the motion compensation operation is performed in the central processor.

In operation 940, one of the computing processors performs a deblocking operation on the multimedia data on which the inverse frequency conversion and dequantization operations are performed.

In one embodiment of the multimedia decoding method, the synchronization signal between the central processor and the computing processors is input and output through a mailbox method.

In one embodiment, data between the memory of the central processor and the memory of the respective computing processors is input and output via the DMA scheme.

In one embodiment, the motion compensation tasks for each data region in the computational processor for motion compensation tasks are synchronized with each other.

In one embodiment, the computational tasks of each of the central processor and the computational processors are processed in parallel in a pipelined structure.

Meanwhile, the above-described embodiments of the present invention can be written as a program that can be executed in a computer, and can be implemented in a general-purpose digital computer that operates the program using a computer-readable recording medium. The computer-readable recording medium may include a magnetic storage medium (eg, ROM, floppy disk, hard disk, etc.), an optical reading medium (eg, CD-ROM, DVD, etc.), and a carrier wave (eg, the Internet). Storage medium).

So far I looked at the center of the preferred embodiment for the present invention. Those skilled in the art will appreciate that the present invention can be implemented in a modified form without departing from the essential features of the present invention. Therefore, the disclosed embodiments should be considered in descriptive sense only and not for purposes of limitation. The scope of the present invention is shown in the claims rather than the foregoing description, and all differences within the scope will be construed as being included in the present invention.

1 illustrates an example of a multicore platform system with a functional partitioning scheme.

2A and 2B show an example of a multicore platform system with a data partitioning scheme.

3A and 3B show differences in the distribution of data and instructions according to application characteristics.

4 is a block diagram of a multi-core platform-based multimedia decoding apparatus according to an embodiment of the present invention.

5 shows a distribution of calculation burdens for each calculation operation.

6 illustrates a hybrid partitioning scheme of a multimedia decoding apparatus according to an embodiment of the present invention.

7 illustrates a queuing method and a communication method between processors according to an embodiment of the present invention.

8 illustrates a processor parallel processing scheme of a pipeline structure according to an embodiment of the present invention.

9 is a flowchart of a multi-core platform-based multimedia decoding method according to an embodiment of the present invention.

Claims (31)

In the multi-core platform-based multimedia decoding method comprising a central processor and a plurality of operational processors, Generating a queue for at least one or more arithmetic operations to be performed by the central processor and the arithmetic processor on the input multimedia data; Performing a motion compensation operation for each data region in which the multimedia data is divided in at least one or more of the computing processors; And Performing a deblocking operation on the multimedia data by one of the computing processors. The method of claim 1, The multimedia decoding method, Reading the input multimedia data in the central processor; Performing an initialization operation for decoding on the read multimedia data in the central processor; Entropy decoding the initialized multimedia data in the central processor; Performing at least one of an inverse frequency transform operation, an inverse quantization operation, and an intra prediction operation on the multimedia data in the central processor; And Storing the multimedia data on which the deblocking operation is performed at the central processor; The queue generating step may include generating a queue for a calculation operation on the entropy-decoded multimedia data. The method of claim 1, wherein the creating a queue comprises: And putting a work parameter block containing work parameters for the arithmetic tasks into a queue for each arithmetic task. The method of claim 3, wherein the creating a queue comprises: And transmitting the work parameter block from the queue for the work to the work processor corresponding to each work queue. The method of claim 4, wherein the transmitting of the job parameter block comprises: Determining a job parameter block that can be processed for a predetermined unit of time by the operation processor; And And transmitting the determined work parameter block to the arithmetic processor every predetermined unit time. The method of claim 1, The multimedia decoding method, Dividing the multimedia data into at least one data area; And Determining a separate computing processor to perform motion compensation operations on each partitioned data region, The queue creation step, When generating a queue for a motion compensation task, a queue including a respective job parameter block to be performed for a corresponding data area in each of the arithmetic processors to perform the motion compensation task is generated. Multimedia decryption method. The method of claim 6, The data region dividing step includes: Calculating an operation burden of an operation processor for a motion compensation operation on the multimedia data; Determining the number of divisions of the multimedia data into a data area in consideration of an operation burden of the operation processor; And Dividing the multimedia data into a data area by the number of divisions; The determining of the operation processor may include determining the same number of operation processors as the number of divisions of the multimedia data. The method of claim 6, wherein dividing the data area, Detecting data dependent on each other among the multimedia data; And And determining the divided data region so that the data which are dependent on each other are not separated. The method of claim 8, In the data region determining step, when the first data among the detected data has a dependency on two or more other data, the data region determination step determines one data among the other dependent data as data to be included in the same data region as the first data. Multimedia decoding method comprising the step of. The method of claim 6, The queue generating step includes storing a job parameter block for each divided data area in a queue for the motion compensation job according to the processing order of the respective data areas, The transmitting of the work parameter block may include transmitting a work parameter block for the respective divided data areas from the queue for the motion compensation work according to the processing order of the respective data areas. Decryption method. The method of claim 1, And a synchronization signal between the central processor and the arithmetic processors is inputted and outputted through a mailbox method. The method of claim 1, And data between the memory of the central processor and the memory of each of the arithmetic processors is inputted and outputted through a direct memory access (DMA) scheme. The method of claim 6, Synchronizing motion compensation tasks for each of the data regions in the operation processor for the motion compensation task. The method of claim 2, And each of the computational tasks of the central processor and the computational processors is processed in parallel in a pipelined structure. The method of claim 14, wherein the parallel processing step comprises: Computation tasks of each of the central processor and the computing processors are, if n is a positive integer greater than 1 and less than N, when the multimedia data is divided into N data units per data processing interval, Performing entropy decoding on the (n-1) th data unit in the central processor and generating the queue; The operation processor for the motion compensation task performs a motion compensation task on the (n-1) th data unit, and the central processor performs an entropy decoding operation on the (n) th data unit and performs the queue. Generating; And The central processor performs an inverse frequency transform operation, an inverse quantization operation, and an in-prediction operation on the (n-1) th data unit, and performs an operation on the (n) th data unit in the arithmetic processor for the motion compensation task. Perform a motion compensation operation on the data processor, perform an entropy decoding operation on the (n + 1) th data unit in the central processor, generate the queue, and perform the operation (n-1) on the operation processor for the deblocking operation. And restoring the (n-1) th data unit by performing a deblocking operation on the) th data unit. In the multi-core platform-based multimedia decoding apparatus comprising a central processor and a plurality of operational processors, A queue generator configured to generate a queue for at least one or more arithmetic operations to be performed by the central processor and the arithmetic processor on the input multimedia data; A motion compensation task performing unit configured to perform a motion compensation task for each data region in which the multimedia data is divided in at least one or more of the computing processors; And And a deblocking task execution unit configured to perform a deblocking operation on the multimedia data by one of the arithmetic processors. The method of claim 16, The multimedia decoding device, The central processor reads the input multimedia data, performs an initialization operation for decoding the read multimedia data, entropy-decodes the initialized multimedia data, and performs an inverse frequency conversion operation on the multimedia data. At least one of performing dequantization and intra prediction, and storing multimedia data on which the deblocking operation is performed. And the queue generator generates a queue for a calculation operation on the entropy-decoded multimedia data. The method of claim 16, wherein the queue generator, And a task parameter block including a task parameter for the arithmetic tasks in a queue for each arithmetic task. The method of claim 18, wherein the queue generator, And a work parameter block transmitter which transmits the work parameter block to the arithmetic processor corresponding to each arithmetic work queue from the queue for the arithmetic work. The method of claim 19, wherein the job parameter block transmission unit, And determining a work parameter block which can be processed by the arithmetic processor for a predetermined unit time, and transmitting the determined work parameter block to the arithmetic processor every predetermined unit time. The method of claim 16, The multimedia decoding device, A data area dividing unit dividing the multimedia data into at least one data area; And A motion compensation processor determiner configured to determine a separate computing processor to perform a motion compensation operation on each divided data area, The queue generator, When generating a queue for a motion compensation task, a queue including a respective job parameter block to be performed for a corresponding data area in each of the arithmetic processors to perform the motion compensation task is generated. Multimedia decoding device. The method of claim 21, The data area dividing unit calculates an operation burden of an operation processor for a motion compensation operation on the multimedia data, and determines the number of divisions of the multimedia data into a data area in consideration of the operation burden of the operation processor. And dividing the multimedia data into a data area by the number of divisions. And the motion compensation processor determiner determines the same number of computation processors as the number of divisions of the multimedia data. The data region dividing unit of claim 21, And detecting the dependent data from among the multimedia data, and determining the divided data area so that the dependent data are not separated from each other. The method of claim 23, The data area determiner may determine one of the other data having the dependency as data to be included in the same data area as the first data when the first data of the detected data has a dependency on two or more other data. Multimedia decoding apparatus, characterized in that. The method of claim 21, The queue generating unit stores a job parameter block for each divided data area in a queue for the motion compensation job according to a processing order of the respective data areas. And the job parameter block transmission unit transmits a job parameter block for each divided data area from the queue for the motion compensation job according to the processing order of the respective data areas. The method of claim 16, And a synchronization signal between the central processor and the arithmetic processors is input / output through a mailbox method. The method of claim 16, And data between the memory of the central processor and the memory of each of the arithmetic processors is inputted and outputted through a direct memory access (DMA) scheme. The method of claim 21, And a motion compensation task for each of the data areas in the operation processor for the motion compensation task. The method of claim 17, And computing operations of the central processor and the computing processors in parallel in a pipeline structure. The method of claim 29, Computation tasks of each of the central processor and the computational processors, Computation tasks of each of the central processor and the computing processors are, if n is a positive integer greater than 1 and less than N, when the multimedia data is divided into N data units per data processing interval, After the central processor performs an entropy decoding operation on the (n-1) th data unit and generates the queue, The operation processor for the motion compensation task performs a motion compensation task on the (n-1) th data unit, and the central processor performs an entropy decoding operation on the (n) th data unit and performs the queue. After creating The central processor performs an inverse frequency transform operation, an inverse quantization operation, and an in-prediction operation on the (n-1) th data unit, and performs an operation on the (n) th data unit in the arithmetic processor for the motion compensation task. Perform a motion compensation operation on the data processor, perform an entropy decoding operation on the (n + 1) th data unit in the central processor, generate the queue, and perform the operation (n-1) on the operation processor for the deblocking operation. And deblocking the (n) th data unit to restore the (n-1) th data unit, thereby being processed in parallel in a pipeline structure. A computer-readable recording medium having recorded thereon a program for implementing the method of any one of claims 1 to 15.

Images