JP4351903B2 - Video encoding device - Google Patents

Description

The present invention uses a plurality of processing units, but about the video encoding KaSo location for coding a moving picture.

  With the advancement of computer technology, a technology for performing video coding and decoding represented by MPEG-2 / MPEG-4 on a software basis has become common. Utilizing this technology, there is known a utilization form in which a personal computer is used as a hard disk recorder by performing image coding / decoding of a television broadcast by software on the personal computer.

  On the other hand, a model has been proposed in which a plurality of high-speed SIMD (Single Instruction Multiple Data) processors are prepared as CPUs installed in next-generation game machines, home servers, digital televisions, etc., and they execute processing in cooperation with each other. ing.

  For example, there is an apparatus in which eight additional processing units (APUs) having the same ISA (Instruction Set Architecture) communicate and process in real time using a shared dynamic random access memory (DRAM) (see Patent Document 1). ).

  According to this, one processor element (PE) is composed of a processing unit (PU), a direct memory access controller (DMAC), and eight APUs. The PU performs processing schedule management and general management of devices. And APU performs a process in parallel according to a schedule.

  This apparatus obtains packetized MPEG data from a TCP / IP network and performs a series of processes for decoding the data. In this series of processing, extraction of MPEG data and decoding of MPEG data are performed by APU. In this way, the processing speed is increased by causing the APU to perform processing that requires a long time.

JP 2002-358289 A

  As described above, a technique for speeding up processing by cooperation of a plurality of APUs is known, but the memory capacity of a local memory included in each APU is very small. Therefore, when performing calculations that consume a large amount of memory or operations that use a large amount of data, it is necessary to exchange information between the APU and the DRAM. Such access from the APU to the DRAM is a major factor in processing delay.

The present invention was made in view of the above, to suppress the delay in processing as described above, and an object thereof is to provide a moving picture coding KaSo location capable of performing encoding processing at high speed.

In order to solve the above-described problems and achieve the object, the present invention is a moving picture coding apparatus for coding a moving picture, instructing the coding process and performing the process related to the coding. A first processor; a second processor that performs processing relating to the encoding based on an instruction from the first processor; a main memory that holds information relating to the encoding; and between the main memory and the first processor And a main memory control means for controlling data exchange between the main memory and the second processor, and a local memory that can be directly accessed from the second processor, wherein the first processor controls the main memory control. The information stored in the main memory is referred to via the means, and the processing related to the encoding includes preprocessing, motion detection processing, motion compensation Management, DCT processing, inverse discrete cosine transform process, a quantization process, the amount of calculation and according to the process and at least one of whole or a part process of variable length coding and syntax generation processing predetermined The second processor performs a first process smaller than the amount of computation, and the second processor refers to the information held in the local memory and, based on an instruction from the first processor, All or part of at least one of processing, motion detection processing, motion compensation processing, DCT processing, inverse discrete cosine transform processing, quantization processing, variable length coding processing, and syntax generation processing, and A second process in which the amount of calculation required for the process is larger than a predetermined amount of calculation is performed, and the entire second process performed by the second processor is divided into a plurality of stages. The main memory further holds information used in each stage, and the local memory is connected via the main memory control means when the second processor executes the second process in one stage. Information used in the stage is acquired from the main memory, and information used in the stage is held.

The moving picture encoding apparatus according to the present invention causes a first processor that can directly access a main memory to perform a process with a relatively large amount of calculation in the encoding process, and a relatively small amount of calculation in the encoding process. This processing is assigned to the second processor that cannot directly access the main memory but can directly access the local memory. As described above, by causing the first processor and the second processor to share a plurality of processes included in the encoding, there is an effect that the processing can be speeded up. Further, when the second processor executes the process in one stage of the second process divided into a plurality of stages, the local memory acquires information used in the stage from the main memory via the main memory control means. By holding the information used in the stage, it is possible to increase the processing speed and reduce the amount of local memory.

It will be described below in detail with reference to examples of video coding KaSo location according to the present invention with reference to the accompanying drawings. Note that the present invention is not limited to the embodiments.

  FIG. 1 is a block diagram showing a hardware configuration of a moving image encoding apparatus 10 according to the present invention. The moving image encoding apparatus 10 includes a processor element 100 and a DRAM (Dynamic Random Access Memory) 110. Furthermore, the processor element 100 has a processing unit 102, a DMAC (Direct Memory Access Controller) 104, and a plurality of additional processing units, that is, a first additional processing unit 106a, a second additional processing unit 106b,. .

  Here, the processing unit 102 and the additional processing unit 106 according to the present embodiment constitute a first processor and a second processor of the present invention, respectively. The DRAM 110 and the DMAC 104 of the present embodiment constitute the main memory and main memory control means of the present invention, respectively.

  The processing unit 102 controls the entire moving image coding apparatus 10. The DRAM 110 holds moving image data to be processed and a program related to encoding processing for the moving image data. The DMAC 104 sends information acquired from the DRAM 110 to the processing unit 102 or the additional processing unit 106. Further, the information acquired from the processing unit 102 and the additional processing unit 106 is sent to the DRAM 110. As described above, the DMAC 104 functions as an interface between the DRAM 110 and the processor element 100. Further, each additional processing unit 106 performs processing related to encoding in accordance with an instruction from the processing unit 102.

  FIG. 2 is a block diagram showing a detailed configuration of the first additional processing unit 106a shown in FIG. The additional processing unit 106a includes a local memory 1060, a register 1062, a first floating point arithmetic unit 1064a, a second floating point arithmetic unit 1064b,..., A first integer arithmetic unit 1066a, and a second integer arithmetic unit 1066b. ..

  The first additional processing unit 106a has a plurality of floating point arithmetic units and a plurality of integer arithmetic units. The first additional processing unit 106a can perform high-speed computation by cooperating with them.

  The local memory 1060 is composed of a relatively small SRAM (Static Random Access Memory) of about 128 kilobytes. The additional processing unit 106 operates using programs and data stored in the local memory 1060. Also, a data transfer request between the DRAM 110 and the additional processing unit 106 is made to the DMAC 104. The additional processing unit 106 cannot directly access the DRAM 110 connected to the DMAC 104.

  When the additional processing unit 106 executes the program, the processing unit 102 controls the DMAC 104, and the stack frame related to the object program is transferred from the DRAM 110 to the local memory 1060 of the additional processing unit 106. Next, the processing unit 102 issues a command for causing the additional processing unit 106 to execute the program. Then, the additional processing unit 106 starts executing the program based on the command issued from the processing unit 102. The additional processing unit 106 also transfers the result of the program to the DRAM 110 via the DMAC 104. When the processing is completed, the additional processing unit 106 instructs the processing unit 102 to generate an interrupt indicating that the processing has been completed.

  Although the detailed configuration of the first additional processing unit 106a has been described with reference to FIG. 2, the detailed configuration of the second additional processing unit 106b and the like is the same as the detailed configuration of the first additional processing unit 106a. .

  FIG. 3 is a flowchart showing an encoding process in the processor element 100. First, the additional processing unit 106 acquires moving image data via the DMAC 104, and performs preprocessing for improving the encoding efficiency (step S100). Next, forward motion detection (step S102) and backward motion detection (step S104) are sequentially performed. Next, motion compensation is performed (step S106). Next, discrete cosine transform is performed on the residual signal obtained by motion compensation (step S108). Next, quantization is performed (step S110), and then variable length coding is performed (step S112).

On the other hand, in order to perform motion detection and motion compensation, after performing quantization (step S110), inverse quantization is performed (step S120), and then inverse discrete cosine transform is performed (step S12 2 ). All of the above processing is performed by the additional processing unit 106.

  As described above, it is desirable to assign processing in units of macroblocks below the picture layer to the additional processing unit 106. The processing below the picture layer is suitable for processing by the additional processing unit 106 because the amount of memory required for processing is relatively small.

  The amount of computation required for motion detection (step S102, step S104), motion compensation (step S106), and DCT processing (step S108) is very large. As described above, it is desirable to assign a process with a large calculation amount, that is, a calculation load to the additional processing unit 106. As described with reference to FIG. 2, the additional processing unit 106 can perform processing at a high speed by the plurality of floating point arithmetic units 1064 and the plurality of integer arithmetic units 1066, so that the processing unit 102 performs processing. Can be processed at a higher speed.

  Note that a threshold value for determining the magnitude of the calculation load may be determined in advance. In this case, it is determined which of the processing unit 102 and the additional processing unit 106 is assigned by comparing the threshold value with the calculation amount.

  As described above, the processing speed can be increased as a whole by defining the processing to be assigned to the additional processing unit 106 and the processing to be assigned to the processing unit 102 with the predetermined calculation amount as a threshold value.

  In addition, the processing below the picture layer is often repeated. Thus, even when there are many repetitive processes, it is suitable for processing by the additional processing unit 106 capable of high-speed processing. In this way, by assigning processes suitable for each function, it is possible to improve the efficiency of the process as a whole.

  In addition to the processing performed by the additional processing unit 106, syntax generation is performed (step S130). The syntax processing (step S130) is performed by the processing unit 102.

  The calculation load for syntax processing is small. However, a large number of tables are used in processing. Therefore, it is desirable to improve processing efficiency by performing processing in cooperation with the DRAM 110 having a large memory capacity. Specifically, the processing unit 102 directly accesses the DRAM 110 via the DMAC 104 and performs syntax processing while referring to a table held in the DRAM 110. As described above, it is desirable to assign a syntax generation higher than the picture layer to the processing unit 102.

  Thus, the encoding process is completed. As described above, the encoding process is shared between the processing unit 102 and the additional processing unit 106, whereby the processing can be performed efficiently and at high speed.

  The encoding process in the processor element 100 is the same as a general MPEG encoding process.

The processing of the additional processing unit 106 described with reference to FIG. 3 is divided into a plurality of stages. In the present embodiment, the processing of the additional processing unit 106 is divided into five stages. That is, the moving image encoding process includes a first stage that performs pre-processing (step S100), a second stage that performs forward motion detection (step S102), and a third stage that performs backward motion detection (step S104). A fourth stage for performing motion compensation (step S106) and DCT processing (step S108), quantization (step S110), variable length coding (step S112), inverse quantization (step S120), and inverse discrete cosine transform A fifth stage for performing (Step S122) is provided.

  During the execution of the first stage processing by the additional processing unit 106, the local memory 1060 holds a program executed in the first stage, moving image data to be processed in the first stage, reference data, and the like. ing. When the processing in the first stage is completed, the program executed in the first stage is saved from the additional processing unit 106 to the DRAM 110 via the DMAC 104.

  Then, a program executed in the second stage is written from the DRAM 110 to the additional processing unit 106 via the DMAC 104. Thus, access to the DRAM 110 is performed only at the timing of switching of each stage, such as switching from the first stage to the second stage and switching from the second stage to the second stage.

  The processing included in each stage is determined based on the size of the local memory 1060 of the additional processing unit 106, the calculation speed in the additional processing unit 106, and the like. In other words, processing of the maximum amount of data that can be held in the memory capacity of the local memory 1060 is set as one stage.

  The encoding process includes a number of complicated processes. Therefore, the data amount of the program for realizing these processes by software is large. In addition, since motion detection and motion compensation target data with a large amount of data such as a reference image, a target image, and a motion compensation image, a large amount of memory is required. Also, variable length coding requires a large amount of memory to perform processing while holding a variable length coding table.

  On the other hand, since the memory capacity of the local memory 1060 is small, the program and data to be used for the encoding process cannot be held in the local memory 1060 at a time. Therefore, it is necessary for the additional processing unit 106 to dynamically acquire a program and data from the DRAM 110 as necessary, and to write the program and data into the DRAM 110.

  However, processing including access to the DRAM 110 takes a long time, which is a main factor that causes processing delay as a whole. Thus, as described above, by minimizing the frequency of accessing the DRAM 110, it is possible to avoid a processing delay due to the access to the DRAM 110.

  Hereinafter, data exchange in each stage will be described with reference to FIGS.

  FIG. 4 is a diagram showing a flow of moving image data in the first stage. In the first stage, first, the DMAC 104 acquires moving image data from the DRAM 110. Then, the DMAC 104 distributes the moving image data to each additional processing unit 106 with a slice as a minimum unit based on an instruction from the processing unit 102. Here, the slice refers to one line in the horizontal direction of the still image constituting the moving image. The distribution of moving image data will be described later.

  Each additional processing unit 106 performs preprocessing (step S200) on the moving image data received from the DMAC 104. The preprocessed moving image data is written back to the DRAM 110 via the DMAC 104.

  Various types of processing are assumed as the preprocessing (step S100). Specifically, there are 4: 2: 2 → 4: 2: 0 conversion, 3: 2 pull-down detection, noise removal, and the like. The program capacity and data capacity for each process differ from one process to another. Therefore, this is one stage.

  In the first stage, as described above, only one stage is used for preprocessing. Therefore, even when the preprocessing requires a large amount of memory, it is not necessary to exchange data with the DRAM 110 during the processing of the first stage, and efficient processing can be performed.

  FIG. 5 is a diagram showing a flow of moving image data in the second stage. In the second stage, each additional processing unit 106 acquires local decoding of moving image data from the DRAM 110 via the DMAC 104. Then, forward motion detection (step S102) is performed. Then, the motion vector obtained by the forward motion detection (step S102) is written back to the DRAM 110 via the DMAC 104.

  The processing in the third stage is the same as the flow of moving image data in the second stage described with reference to FIG.

  FIG. 6 is a diagram showing a flow of moving image data in the fourth stage. In the fourth stage, each additional processing unit 106 acquires moving image data, a locally decoded image, and a motion vector from the DRAM 110 via the DMAC 104. The additional processing unit 106 performs motion compensation on the acquired moving image data, and the local memory 1060 holds a residual signal obtained by motion compensation. Further, DCT processing (step S108) is performed on the residual signal held in the local memory 1060. The result obtained by the DCT process, that is, the DCT coefficient is written back to the DRAM 110.

  In the motion compensation (S106), moving image data, a local decoded image, a motion vector, and a residual signal are targeted. The data capacity of the data to be processed is large and occupies most of the capacity of the local memory 1060. On the other hand, in the DCT process (step S108), the data area secured in the motion compensation (S106) can be used as a storage destination of DCT coefficients after the DCT process (step S108). Further, the DCT process (step S108) program itself is small. Therefore, these processes are combined into one stage.

  FIG. 7 is a diagram showing a flow of moving image data in the fifth stage. In the fifth stage, each additional processing unit 106 acquires a DCT coefficient from the DRAM 110 via the DMAC 104. The additional processing unit 106 performs quantization (step S110) on the acquired DCT coefficient. The local memory 1060 holds the DCT coefficient after quantization.

  Further, each additional processing unit 106 acquires a motion vector from the DRAM 110 via the DMAC 104. Then, the additional processing unit 106 performs variable length coding (step S112) on the motion vector and the quantized DCT coefficient held in the local memory 1060, respectively.

  Also, the quantized DCT coefficient is read from the local memory 1060, and inverse quantization (step S120) is performed. Then, the local memory 1060 holds the result. Further, the additional processing unit 106 performs inverse discrete cosine transform (step S122) on the DCT coefficients after inverse quantization held in the local memory 1060, and creates a local decoded image. Then, the local decoded image and the variable length encoded data are written into the DRAM 110.

Quantization (step S110), variable-length coding (step S112), inverse quantization (step S120), and inverse discrete cosine transform (step S122) all have a small program amount and data capacity. However, the data amount of the table used in variable length coding (step S112) is relatively large. Therefore, these processes are combined into one stage.

  FIG. 8 is a diagram for explaining the assignment of processing to each additional processing unit 106. As shown in FIG. 8, the additional processing unit 106 is set with a time budget for controlling the timing. Each additional processing unit 106 performs the same processing on different slices in the same time budget. Here, the slice refers to one line in the horizontal direction of the still image constituting the moving image.

  For example, the process for slice 1 is assigned to the first additional processing unit 106a, and the process for slice 2 is assigned to the second additional processing unit 106b. As described above, the plurality of additional processing units 106 perform the parallel processing by sharing the processing for one moving image data.

  For example, in the first stage, the first additional processing unit 106a performs preprocessing on slice 1. Then, the slice 1 after the preprocessing is written back to the DRAM 110. The second additional processing unit 106b performs preprocessing for slice 2. Then, the slice 2 after the preprocessing is written back to the DRAM 110. Similarly, each additional processing unit 106 performs preprocessing and writes the result back to the DRAM 110.

  As described above, the plurality of additional processing units 106 share processing in units of slices, so that processing speed can be increased.

  When the number of slices is larger than the number of additional processing units 106, a plurality of slices may be assigned to one additional processing unit. For example, slices 1 to 3 are assigned to the first additional processing unit 106a, and slices 4 to 6 are assigned to the second additional processing unit 106b.

  Furthermore, when the first additional processing unit 106a completes the first stage processing, the second additional processing unit 106b may not complete the first stage processing. For example, the second additional processing unit 106b may be processing the slice 4 when the first additional processing unit 106a completes the first stage processing. For example, even when the slices are the same, the amount of computation required for processing is different.

  In this case, the first additional processing unit 106a performs processing on the slice 6 while the second additional processing unit 106b processes the slice 4. As a result, the processing speed can be increased as compared with the case where the second additional processing unit 106b performs processing for slice 4 to slice 6. As described above, when the processing speeds of the additional processing units 106 are different, the processing can be further speeded up by redistributing the slices.

  In the present embodiment, moving image data is allocated to each additional processing unit 106 in slice units, but the unit of moving image data allocation to each additional processing unit 106 is not limited to this. There may be other units. For example, moving image data may be allocated to each additional processing unit 106 in units of macro blocks that are finer units constituting a slice.

  FIG. 9 is a diagram for explaining the processing of each additional processing unit 106 when the additional processing unit 106 is in charge of different processing. As shown in FIG. 9, if each additional processing unit 106 is in charge of different processing, the second additional processing unit 106 a while the first additional processing unit 106 a is performing the first stage processing on slice 1. 106b cannot perform the second stage processing on slice 1 and enters a standby state. As described above, if the plurality of additional processing units 106 perform different processes, the other additional processing units 106 may not be able to start the processing until the processing in one additional processing unit 106 is completed, which is inefficient. If a plurality of additional processing units 106 are in charge of different processes, sufficient processing efficiency cannot be achieved despite parallel processing performed by the plurality of additional processing units 106.

  Therefore, as described with reference to FIG. 8, each additional processing unit 106 performs the same processing on different slices. As a result, frequent occurrence of the standby state in each additional processing unit 106 as described with reference to FIG. 9 can be avoided, and processing efficiency can be improved.

  As described above, the moving image encoding apparatus and the encoding method according to the present invention are useful for encoding moving image data, and particularly useful for encoding moving image data using a plurality of processing units. .

It is a block diagram which shows the hardware constitutions of the moving image encoder 10 of this invention. It is a block diagram which shows the detailed structure of the 1st addition process unit 106a shown in FIG. 3 is a flowchart showing an encoding process in a processor element 100. It is a figure which shows the flow of the moving image data in a 1st stage. It is a figure which shows the flow of the moving image data in a 2nd stage. It is a figure which shows the flow of the moving image data in a 4th stage. It is a figure which shows the flow of the moving image data in a 5th stage. FIG. 6 is a diagram for explaining process assignment to each additional processing unit 106; It is a figure for demonstrating the process of each additional process unit 106 in case the additional process unit 106 takes charge of a different process, respectively.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Moving image encoder 100 Processor element 102 Processing unit 106 Additional processing unit 1060 Local memory 1062 Register 1064 Floating point arithmetic unit 1066 Integer arithmetic unit

Claims (5)

A moving image encoding device for encoding a moving image,
A first processor for instructing the encoding process and performing the encoding process;
A second processor that performs processing related to the encoding based on an instruction from the first processor;
A main memory holding information about the encoding;
Main memory control means for controlling data exchange between the main memory and the first processor and between the main memory and the second processor;
A local memory directly accessible from the second processor,
The first processor refers to information held in the main memory via the main memory control means, and as processing related to the encoding, preprocessing, motion detection processing, motion compensation processing, DCT processing, inverse discrete cosine, A first process that is all or a part of at least one of a conversion process, a quantization process, a variable-length encoding process, and a syntax generation process, and the amount of calculation for the process is smaller than a predetermined amount of calculation And
The second processor refers to the information held in the local memory and, based on an instruction from the first processor, performs processing related to the encoding as preprocessing, motion detection processing, motion compensation processing, and DCT processing. , An inverse discrete cosine transform process, a quantization process, a variable length encoding process, and a syntax generation process. The second process is also large,
The entire second process performed by the second processor is divided into a plurality of stages,
The main memory further holds information used in each stage,
The local memory acquires information used in the stage from the main memory via the main memory control means when the second processor executes the second process in the one stage. A moving picture coding apparatus characterized by holding information to be used. The moving image according to claim 1, wherein the local memory saves information used in the stage to the main memory when the second processor completes the second processing in the one stage. Image encoding device. Of the second processing, the pre-processing is included in the first stage, out of the motion detection processing, the forward motion detection processing is included in the second stage, and out of the motion detection processing, the backward motion detection processing is the third stage. The motion compensation processing and DCT processing are included in a fourth stage, and the inverse discrete cosine transform processing, quantization processing, and variable length encoding processing are included in a fifth stage. Or the moving image encoding apparatus of 2. A plurality of the second processors;
  The first processor divides the moving image into a plurality of partial data, distributes the moving image to the plurality of second processors in units of the partial data,
  4. The moving picture coding apparatus according to claim 1, wherein the plurality of second processors perform second processing on the partial data received from the first processor. 5. . 5. The first processor according to claim 4, wherein the first processor divides a still image constituting the moving image into a plurality of slices, and distributes the moving image to the plurality of second processors in units of the slices. Video encoding device.

Images