JP4398717B2 - Video decoding device - Google Patents

Description

The present invention uses a plurality of processing units, but about the video decoding KaSo location for decoding a moving image.

  With the advancement of computer technology, a technology for performing video coding and video decoding represented by MPEG-2 / MPEG-4 on a software basis has become common. Utilizing this technique, a utilization form is known in which a television broadcast is decoded by software on a personal computer and the personal computer is used as a hard disk recorder.

  On the other hand, a model has been proposed in which a plurality of high-speed SIMD (Single Instruction Multiple Data) processors are prepared as CPUs installed in next-generation game machines, home servers, digital televisions, etc., and they execute processing in cooperation with each other. ing.

  For example, there is an apparatus in which eight additional processing units (APUs) having the same ISA (Instruction Set Architecture) communicate and process in real time using a shared dynamic random access memory (DRAM) (see Patent Document 1). ).

  According to this, one processor element (PE) is composed of a processing unit (PU), a direct memory access controller (DMAC), and eight APUs. The PU performs processing schedule management and general management of devices. And APU performs a process in parallel according to a schedule.

  This apparatus obtains packetized MPEG data from a TCP / IP network and performs a series of processes for decoding the data. In this series of processing, extraction of MPEG data and decoding of MPEG data are performed by APU. In this way, the processing speed is increased by causing the APU to perform processing that requires a long time.

JP 2002-358289 A

  As described above, a technique for speeding up processing by cooperation of a plurality of APUs is known, but the memory capacity of a local memory included in each APU is very small. Therefore, when performing calculations that consume a large amount of memory or operations that use a large amount of data, it is necessary to exchange information between the APU and the DRAM. Such access from the APU to the DRAM is a major factor in processing delay.

The present invention was made in view of the above, to suppress the delay in processing as described above, and an object thereof is to provide a moving picture decoding KaSo location capable of performing a decoding process at high speed.

In order to solve the above-described problems and achieve the object, the present invention is a moving picture decoding apparatus for decoding a moving picture, instructing the decoding process and performing the process related to the decoding. A first processor; a second processor that performs processing related to decoding based on an instruction from the first processor; a main memory that holds information related to decoding; and between the main memory and the first processor And a main memory control means for controlling data exchange between the main memory and the second processor, and a local memory that can be directly accessed from the second processor, wherein the first processor controls the main memory control. The information stored in the main memory is referred to via the means, and as the processing related to the decoding, syntax analysis processing, variable length recovery Processing, inverse quantization processing, inverse discrete cosine transform process, the amount of computation of the amount of computation according to and a motion compensation process and all or part of the processing of at least one of the post-treatment process is predetermined from The second processor refers to the information held in the local memory, and based on an instruction from the first processor, a syntax analysis process as a process related to the decoding, All or part of at least one of variable-length decoding processing, inverse quantization processing, inverse discrete cosine transform processing, motion compensation processing, and post-processing, and the amount of computation related to the processing is determined in advance A second process larger than the amount of computation is performed, and the entire second process performed by the second processor is divided into a plurality of stages, and the main memory is used in each stage. The local memory is information used in the stage from the main memory via the main memory control means when the second processor executes the second process in one stage. And information used in the stage is held.

The moving picture decoding apparatus according to the present invention causes a first processor that can directly access a main memory to perform a process with a relatively large calculation amount in the decoding process, and a relatively small calculation amount in the decoding process. This processing is assigned to the second processor that cannot directly access the main memory but can directly access the local memory. In this way, by causing the first processor and the second processor to share a plurality of processes included in the decoding, there is an effect that the processing can be speeded up. Further, when the second processor executes the process in one stage of the second process divided into a plurality of stages, the local memory acquires information used in the stage from the main memory via the main memory control means. By holding the information used in the stage, it is possible to increase the processing speed and reduce the amount of local memory.

It will be described below in detail with reference to examples of a video decoding KaSo location according to the present invention with reference to the accompanying drawings. Note that the present invention is not limited to the embodiments.

  FIG. 1 is a block diagram showing a hardware configuration of a moving picture decoding apparatus 10 according to the present invention. The moving picture decoding apparatus 10 includes a processor element 100 and a DRAM (Dynamic Random Access Memory) 110. Furthermore, the processor element 100 has a processing unit 102, a DMAC (Direct Memory Access Controller) 104, and a plurality of additional processing units, that is, a first additional processing unit 106a, a second additional processing unit 106b,. .

  Here, the processing unit 102 and the additional processing unit 106 according to the present embodiment constitute a first processor and a second processor of the present invention, respectively. The DRAM 110 and the DMAC 104 of the present embodiment constitute the main memory and main memory control means of the present invention, respectively.

  The processing unit 102 controls the entire moving image coding apparatus 10. The DRAM 110 holds moving image data to be processed and a program related to encoding processing for the moving image data. The DMAC 104 sends information acquired from the DRAM 110 to the processing unit 102 or the additional processing unit 106. Further, the information acquired from the processing unit 102 and the additional processing unit 106 is sent to the DRAM 110. As described above, the DMAC 104 functions as an interface between the DRAM 110 and the processor element 100. In addition, each additional processing unit 106 performs processing related to decoding according to an instruction from the processing unit 102.

  FIG. 2 is a block diagram showing a detailed configuration of the first additional processing unit 106a shown in FIG. The additional processing unit 106a includes a local memory 1060, a register 1062, a first floating point arithmetic unit 1064a, a second floating point arithmetic unit 1064b,..., A first integer arithmetic unit 1066a, and a second integer arithmetic unit 1066b. ..

  The first additional processing unit 106a has a plurality of floating point arithmetic units and a plurality of integer arithmetic units. The first additional processing unit 106a can perform high-speed computation by cooperating with them.

  The local memory 1060 is composed of a relatively small SRAM (Static Random Access Memory) of about 128 kilobytes. The additional processing unit 106 operates using programs and data stored in the local memory 1060. Also, a data transfer request between the DRAM 110 and the additional processing unit 106 is made to the DMAC 104. The additional processing unit 106 cannot directly access the DRAM 110 connected to the DMAC 104.

  When the additional processing unit 106 executes the program, the processing unit 102 controls the DMAC 104, and the stack frame related to the object program is transferred from the DRAM 110 to the local memory 1060 of the additional processing unit 106. Next, the processing unit 102 issues a command for causing the additional processing unit 106 to execute the program. Then, the additional processing unit 106 starts executing the program based on the command issued from the processing unit 102. The additional processing unit 106 also transfers the result of the program to the DRAM 110 via the DMAC 104. When the processing is completed, the additional processing unit 106 instructs the processing unit 102 to generate an interrupt indicating that the processing has been completed.

  Although the detailed configuration of the first additional processing unit 106a has been described with reference to FIG. 2, the detailed configuration of the second additional processing unit 106b and the like is the same as the detailed configuration of the first additional processing unit 106a. .

  FIG. 3 is a flowchart showing a decoding process in the processor element 100. First, the processing unit 102 analyzes an MPEG syntax for performing variable length decoding described later (step S100). Independently of this, the additional processing unit 106 acquires moving image data via the DMAC 104 and performs variable length decoding (step S110). Thereby, binary data is decoded. Next, inverse quantization is performed to obtain DCT coefficients (step S112). Next, an inverse discrete cosine transform is performed on the DCT coefficient to obtain a residual signal (step S114). Next, a decoded image is created by motion compensation based on the residual signal and the motion vector obtained in the variable length decoding process (step S110) (step S116). Next, post-processing is performed (step S118). Note that the post-processing is processing based on visual characteristics and improves subjective image quality.

  This completes the decoding process. As described above, by sharing the decoding process to the processing unit 102 and the additional processing unit 106, the processing can be performed efficiently and at high speed.

  In addition, it is desirable to assign MPEG syntax analysis to the processing unit 102. The calculation load in MPEG syntax analysis is small. But it uses a lot of tables. Therefore, by making the processing unit 102 that can directly access the large-capacity DRAM 110 via the DMAC 104 perform the above processing, the processing efficiency can be improved as compared with the case where the syntax analysis is assigned to the additional processing unit 104. As described above, it is desirable to assign processing higher than the picture layer to the processing unit 102. This is because the processing above the picture layer has a small amount of processing, so that it is difficult to delay the processing as a whole even if the processing speed is low.

  In addition, it is desirable to assign processing in units of macroblocks below the picture layer to the additional processing unit 106. Processing below the picture layer is suitable for processing by the additional processing unit 106.

  For example, the inverse discrete cosine transform (step S104) and motion compensation (step S106) have a large calculation amount. As described above, it is desirable to assign a process with a large calculation amount, that is, a calculation load to the additional processing unit 106. As described with reference to FIG. 2, the additional processing unit 106 can perform processing at a high speed by the plurality of floating point arithmetic units 1064 and the plurality of integer arithmetic units 1066, so that the processing unit 102 performs processing. This is because processing can be performed at a higher speed.

  Note that a threshold value for determining the magnitude of the calculation load may be determined in advance. In this case, it is determined which of the processing unit 102 and the additional processing unit 106 is assigned by comparing the threshold value with the calculation amount.

  As described above, the processing speed can be increased as a whole by defining the processing to be assigned to the additional processing unit 106 and the processing to be assigned to the processing unit 102 with the predetermined calculation amount as a threshold value.

  In addition, the processing below the picture layer is often repeated. Thus, even when there are many repetitive processes, it is suitable for processing by the additional processing unit 106 capable of high-speed processing. In this way, by assigning processes suitable for each function, it is possible to improve the efficiency of the process as a whole.

  The decoding process in the processor element 100 is the same as a general MPEG decoding process.

  The processing of the additional processing unit 106 described with reference to FIG. 3 is divided into a plurality of stages. In the present embodiment, the processing of the additional processing unit 106 is divided into three stages. That is, the decoding process includes a first stage for performing variable length decoding (step S110) and inverse quantization (step S112), a second stage for performing inverse discrete cosine transform output (step S114), and motion compensation (step S116). A stage and a third stage for performing post-processing (step S118);

  During the execution of the first stage processing by the additional processing unit 106, the local memory 1060 holds a program executed in the first stage, moving image data to be processed in the first stage, reference data, and the like. ing. When the processing in the first stage is completed, the program executed in the first stage is saved from the additional processing unit 106 to the DRAM 110 via the DMAC 104.

  Then, a program executed in the second stage is written from the DRAM 110 to the additional processing unit 106 via the DMAC 104. Thus, access to the DRAM 110 is performed only at the timing of switching from the first stage to the second stage, switching from the second stage to the second stage, that is, switching of each stage.

  The processing included in each stage is determined based on the size of the local memory 1060 of the additional processing unit 106, the calculation speed in the additional processing unit 106, and the like. In other words, processing of the maximum amount of data that can be held in the memory capacity of the local memory 1060 is set as one stage.

  The decoding process includes a number of complicated processes. Therefore, the data amount of the program for realizing these processes by software is large. In addition, since motion detection and motion compensation target data with a large amount of data such as a reference image, a target image, and a motion compensation image, a large amount of memory is required. Also, variable length decoding requires a large amount of memory to perform processing while holding a variable length decoding table.

  On the other hand, since the memory capacity of the local memory 1060 is small, the program and data to be used for the encoding process cannot be held in the local memory 1060 at a time. Therefore, it is necessary for the additional processing unit 106 to dynamically acquire a program and data from the DRAM 110 as necessary, and to write the program and data into the DRAM 110.

  However, processing including access to the DRAM 110 takes a long time, which is a main factor that causes processing delay as a whole. Thus, as described above, by minimizing the frequency of accessing the DRAM 110, it is possible to avoid a processing delay due to the access to the DRAM 110.

  Hereinafter, data exchange in each stage will be described with reference to FIGS.

  FIG. 4 is a diagram illustrating a flow of binary data in the first stage. In the first stage, first, the DMAC 104 acquires binary data from the DRAM 110. Then, the DMAC 104 distributes the binary data to each additional processing unit 106 with a slice as a minimum unit based on an instruction from the processing unit 102. Here, the slice is one line of the moving image in the horizontal direction of the still image. The distribution of binary data will be described later.

  Each additional processing unit 106 performs variable length decoding (step S110) on the binary data received from the DMAC 104, and the local memory 1060 holds the quantized DCT coefficients obtained by the variable length decoding. Further, inverse quantization (step S112) is performed on the quantized DCT coefficients held in the local memory 1060. Then, the result obtained by inverse quantization, that is, the DCT coefficient is written back to the DRAM 110 via the DMAC 104.

  In the variable length decoding (step S110), the variable length decoding table is referred to. The data capacity of this table is large and almost occupies the local memory 1060. On the other hand, the inverse quantization (step S112) program and the amount of data required for this are small. Thus, inverse quantization can be included in the same stage as variable length decoding. Therefore, these are combined into one stage.

  FIG. 5 is a diagram illustrating a flow of binary data in the second stage. In the second stage, first, the additional processing unit 106 acquires the DCT coefficient obtained in the first stage from the DRAM 110 via the DMAC 104. Then, the additional processing unit 106 performs inverse discrete cosine transform (step S114) on the DCT coefficient, and the local memory 1060 holds the residual signal obtained by the inverse discrete cosine transform. Further, the additional processing unit 106 obtains a motion vector and a reference decoded image from the DRAM 110, and performs motion compensation (step S116) together with the residual signal held in the local memory 1060. Then, the created decoded image is written into the DRAM 110 via the DMAC 104.

  The decoded image and the residual signal to be processed in the inverse discrete cosine transform (step S114) and the motion compensation (step S116) are not necessary depending on the decoding mode of each macroblock, and therefore, the amount of data required for processing is larger than that of the encoder. There are few. Therefore, these processes are combined into one stage.

  FIG. 6 is a diagram showing a flow of binary data in the third stage. In the third stage, the additional processing unit 106 acquires the decoded image obtained in the second stage from the DRAM 110 via the DMAC 104. Then, post-processing (step S118) is performed on the decoded image to improve the subjective image quality. Then, the processed image is written into the DRAM 110 via the DMAC 104. As post-processing, various types of processing are assumed. And the capacity | capacitance and data capacity of the program concerning each process differ for every process. Therefore, this is one stage.

  In the third stage, as described above, only one post-processing is performed. Therefore, even when the preprocessing requires a large amount of memory, it is not necessary to exchange data with the DRAM 110 during the processing related to the third stage, and efficient processing can be performed.

  FIG. 7 is a diagram for explaining the assignment of processing to each additional processing unit 106. As shown in FIG. 7, a time budget for controlling timing is set in the additional processing unit 106. Each additional processing unit 106 performs the same processing on different slices in the same time budget. Here, the slice refers to one line in the horizontal direction of the still image constituting the moving image.

  For example, the process for slice 1 is assigned to the first additional processing unit 106a, and the process for slice 2 is assigned to the second additional processing unit 106b. As described above, the plurality of additional processing units 106 perform the parallel processing by sharing the processing for one moving image data.

  For example, in the first stage, the first additional processing unit 106a performs preprocessing on slice 1. Then, the slice 1 after the preprocessing is written back to the DRAM 110. The second additional processing unit 106b performs preprocessing for slice 2. Then, the slice 2 after the preprocessing is written back to the DRAM 110. Similarly, each additional processing unit 106 performs preprocessing and writes the result back to the DRAM 110.

  As described above, the plurality of additional processing units 106 share processing in units of slices, so that processing speed can be increased.

  When the number of slices is larger than the number of additional processing units 106, a plurality of slices may be assigned to one additional processing unit. For example, slices 1 to 3 are assigned to the first additional processing unit 106a, and slices 4 to 6 are assigned to the second additional processing unit 106b.

  Furthermore, when the first additional processing unit 106a completes the first stage processing, the second additional processing unit 106b may not complete the first stage processing. For example, the second additional processing unit 106b may be processing the slice 4 when the first additional processing unit 106a completes the first stage processing. For example, even when the slices are the same, the amount of computation required for processing is different.

  In this case, the first additional processing unit 106a performs processing on the slice 6 while the second additional processing unit 106b processes the slice 4. As a result, the processing speed can be increased as compared with the case where the second additional processing unit 106b performs processing for slice 4 to slice 6. As described above, when the processing speeds of the additional processing units 106 are different, the processing can be further speeded up by redistributing the slices.

  In the present embodiment, moving image data is allocated to each additional processing unit 106 in slice units, but the unit of moving image data allocation to each additional processing unit 106 is not limited to this. There may be other units. For example, moving image data may be allocated to each additional processing unit 106 in units of macro blocks that are finer units constituting a slice. As another example, moving image data may be assigned to each additional processing unit 106 in units of video packets divided in moving image data starting from the resynchronization marker.

  FIG. 8 is a diagram for explaining processing in which the processing unit 102 allocates binary data to the additional processing unit 106. The processing unit 102 detects the slice start code 310 from the binary data acquired from the DRAM 110 via the DMAC 104. Data from one slice start code 310 a to the next slice start code 310 b described above is determined as the slice code 312. As described above, the processing unit 102 divides binary data into a plurality of data based on the slice start code, and assigns the data to each additional processing unit 106.

  FIG. 9 is a diagram for explaining the processing of each additional processing unit 106 when the additional processing unit 106 is in charge of different processing. As shown in FIG. 9, if each additional processing unit 106 is in charge of different processing, the second additional processing unit 106 a while the first additional processing unit 106 a is performing the first stage processing on slice 1. 106b cannot perform the second stage processing on slice 1 and enters a standby state. As described above, if a plurality of additional processing units 106 perform different processes, the other additional processing units 106 may not be able to start processing until the processing in one additional processing unit 106 is completed, resulting in poor efficiency. If a plurality of additional processing units 106 are in charge of different processes, sufficient processing efficiency cannot be achieved despite parallel processing performed by the plurality of additional processing units 106.

  Therefore, as described with reference to FIG. 7, each additional processing unit 106 performs the same processing on different slices. As a result, frequent occurrence of the standby state in each additional processing unit 106 as described with reference to FIG. 9 can be avoided, and processing efficiency can be improved.

As described above, the moving picture decoding KaSo location according to the present invention is useful for decoding the moving picture data, in particular, useful using a plurality of processing units, the processing of decoding the binary data to the moving image data It is.

3 is a block diagram illustrating a hardware configuration of a moving picture decoding apparatus 10. FIG. It is a block diagram which shows the detailed structure of the 1st addition process unit 106a shown in FIG. 4 is a flowchart showing a decoding process in the processor element 100. It is a figure which shows the flow of the binary data in a 1st stage. It is a figure which shows the flow of the binary data in a 2nd stage. It is a figure which shows the flow of the binary data in a 3rd stage. FIG. 6 is a diagram for explaining process assignment to each additional processing unit 106; FIG. 10 is a diagram for explaining processing in which a processing unit assigns binary data to an additional processing unit. It is a figure for demonstrating the process of each additional process unit 106 in case the additional process unit 106 takes charge of a different process, respectively.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Moving image decoding apparatus 100 Processor element 102 Processing unit 106 Additional processing unit 1060 Local memory 1062 Register 1064 Floating point arithmetic unit 1066 Integer arithmetic unit

Claims (7)

A video decoding device for decoding a video,
A first processor for instructing the decoding process and performing a process related to the decoding;
A second processor that performs processing related to the decoding based on an instruction from the first processor;
A main memory for holding information relating to the decoding;
Main memory control means for controlling data exchange between the main memory and the first processor and between the main memory and the second processor;
A local memory directly accessible from the second processor,
The first processor refers to information held in the main memory via the main memory control means, and as processing related to the decoding, syntax analysis processing, variable length decoding processing, inverse quantization processing, inverse processing Performing a first process that is all or a part of at least one of the discrete cosine transform process, the motion compensation process, and the post-process, and in which the calculation amount for the process is smaller than a predetermined calculation amount;
The second processor refers to information held in the local memory, and, based on an instruction from the first processor, as processing related to the decoding, syntax analysis processing, variable length decoding processing, inverse quantum A second process that is all or a part of at least one of a conversion process, an inverse discrete cosine transform process, a motion compensation process, and a post-process, and the amount of calculation for the process is larger than a predetermined amount of calculation Done
The entire second process performed by the second processor is divided into a plurality of stages,
The main memory further holds information used in each stage,
The local memory acquires information used in the stage from the main memory via the main memory control means when the second processor executes the second process in the one stage. A moving picture decoding apparatus characterized by holding information to be used.   The moving image according to claim 1, wherein the local memory saves information used in the stage to the main memory when the second processor completes the second processing in the one stage. Image decoding device. Among the second processes, the variable length decoding process and the inverse quantization process are included in a first stage, the inverse discrete cosine transform process and the motion compensation process are included in a second stage, and the post-processing is the moving picture decoding apparatus according to claim 1 or 2, characterized in that included in the third stage. A plurality of the second processors;
The first processor divides the moving image into a plurality of partial data, distributes the moving image to the plurality of second processors in units of the partial data,
It said plurality of second processors, the moving picture decoding apparatus according to any one of claims 1 to 3, characterized in that the second processing respectively the partial data received from the first processor . 5. The moving image according to claim 4 , wherein the first processor detects a start code included in the partial data, and distributes the moving image to the plurality of second processors based on the start code. Image decoding device. Wherein the first processor is a still image constituting the moving image is divided into a plurality of slices, according to the moving image in the slice units to claim 4, characterized in that allocated to the plurality of second processors Video decoding device. 5. The moving picture decoding according to claim 4 , wherein the first processor divides the moving picture into a plurality of video packets and distributes the moving picture to the plurality of second processors in units of the video packets. 6. Device.

Images