JP2009153020A - Data processing apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a data processing apparatus accelerating network abstraction layering (NAL) processing without requiring expansion in memory bus band or improvement in operation frequency. <P>SOLUTION: A NAL device 2 is provided with: a data processing section 20 inputting a payload of encoded image data from an encoding device 1; and a data buffer 23 storing therein a payload of non-image data input via a bus 7 under control of a CPU 5. A predetermined NAL header is input to the data processing section 20 via the bus 7 under the control of the CPU 5. The data processing section 20 appends the NAL header to the payload of non-image data read from the data buffer 23 to produce a first NAL unit 60, appends the NAL header to the payload of image data to produce a second NAL unit 70, and arrays the first NAL unit 60 and the second NAL unit 70 to produce an access unit 90. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a data processing apparatus. The present invention relates to a NAL device in the H.264 standard.

  H. In the H.264 standard, NAL processing (NAL: Network Abstraction Layer) is performed between encoding processing for encoding a moving image and PES processing for multiplexing a bit stream into a PES packet (PES: Packetized Elementary Stream). It is prescribed. In the NAL processing, NAL units are generated from encoded data and parameter sets, and an access unit is generated by arranging a plurality of NAL units in a predetermined order.

  In a conventional system, NAL processing is performed by software processing using a DSP (Digital Signal Processor) or CPU. In order to increase the processing speed in software processing, it is necessary to expand the memory bus bandwidth and improve the operating frequency.

  H. A technique related to the NAL processing of the H.264 standard is disclosed in, for example, Patent Document 1 below.

JP 2005-203950 A

  As described above, in order to increase the processing speed in software processing, it is necessary to expand the memory bus band and improve the operating frequency. However, the expansion of the memory bus band causes an increase in memory capacity, which causes disadvantages in system design such as an increase in the number of parts. In addition, since an increase in operating frequency causes an increase in power consumption, there are also disadvantages in system design.

  The present invention has been made in view of such circumstances, and provides a data processing apparatus capable of realizing high-speed NAL processing without requiring expansion of a memory bus band or improvement of an operating frequency. With the goal.

  According to a first aspect of the present invention, there is provided a data processing unit capable of inputting a payload of image data encoded by an external encoding device from the encoding device, and a payload of non-image data as an external control device. And a data buffer capable of storing the payload of the input non-image data, and the data processing unit includes a predetermined NAL (Network Abstraction Layer) header, Input is possible via the external bus under the control of the control device, and the data processing unit supports non-image data by adding the NAL header to the non-image data payload read from the data buffer. Generating a first NAL unit, and adding the NAL header to the payload of the image data. A second NAL unit corresponding to image data is generated, and an access unit is generated by arranging the first NAL unit and the second NAL unit.

  A data processing device according to a second invention is the data processing device according to the first invention, in particular, the data processing unit has a register capable of storing a plurality of NAL units, and the data processing unit generates By storing the first NAL unit and the second NAL unit in this order in the register, and outputting the first NAL unit and the second NAL unit in this order from the register, the access unit Is generated.

  According to a third aspect of the present invention, in the data processing apparatus according to the first or second aspect of the present invention, the dummy data can be stored in the data buffer. The filler NAL can be generated by repeatedly reading data from the data buffer.

  According to the first to third aspects of the data processing device, since the data processing device as the NAL device is configured by hardware, the NAL is not accompanied by expansion of the memory bus band or improvement of the operating frequency. It is possible to realize high-speed processing. In addition, by executing the non-image data payload input processing and the NAL header input processing by software processing under the control of an external control device, compared to a case where all processing is realized by a hardware configuration, the device It is possible to simplify the configuration.

  In particular, according to the data processing device of the second invention, it is possible to easily realize access unit generation processing by arranging a plurality of NAL units in a predetermined order by cooperation of hardware configuration and software processing. Is possible.

  In particular, according to the data processing device of the third invention, the filler NAL generation processing can be easily realized by the cooperation of the hardware configuration and the software processing. In addition, by repeatedly reading the dummy data from the data buffer, it is possible to generate a filler NAL having a capacity larger than the capacity of the data buffer.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In addition, the element which attached | subjected the same code | symbol in different drawing shall show the same or corresponding element.

  FIG. 1 is a block diagram illustrating a part of a configuration of an image processing system compliant with the H.264 standard. As shown in FIG. 1, the image processing system includes an encoding device 1, a NAL device 2, a PES device 3, a memory 4 such as a DRAM, and a CPU 5 (control device). The encoding device 1, the NAL device 2, the PES device 3, and the memory 4 are connected to the bus 6. The encoding device 1, NAL device 2, PES device 3, and CPU 5 are connected to a bus 7.

  The encoding device 1 generates encoded data by performing a moving image encoding process. The NAL device 2 performs NAL unit generation processing and access unit generation processing based on the encoded data. The PES device 3 performs a process of multiplexing the access unit into the PES packet.

  FIG. 2 is a block diagram showing a specific configuration of the NAL device 2 shown in FIG. As shown in FIG. 2, the NAL device 2 includes interfaces 10 to 14, a data processing unit 20, a DMA buffer 21 (DMA: Direct Memory Access), a controller 22, and a data buffer 23.

  A data processing unit 20 is connected to the output of the data buffer 23, and a DMA buffer 21 is connected to the output of the data processing unit 20. The controller 22 is connected to the data processing unit 20, the DMA buffer 21, and the data buffer 23, respectively, and controls these operations.

  A data processing unit 20 and a controller 22 are connected to the interface 10 inside the NAL device 2. In addition, the encoding device 1 is connected to the interface 10 outside the NAL device 2. Therefore, the data processing unit 20 is connected to the encoding device 1 via the interface 10. As a modification, the bus 6 may be connected to the interface 10 outside the NAL device 2. In the case of this modification, the data processing unit 20 is connected to the memory 4 (and the encoding device 1) via the interface 10 and the bus 6.

  A DMA buffer 21 is connected to the interface 11 in the NAL device 2. Further, a bus 6 is connected to the interface 11 outside the NAL device 2. Accordingly, the DMA buffer 21 is connected to the memory 4 (and the PES device 3) via the interface 11 and the bus 6.

  A controller 22 is connected to the interface 12 inside the NAL device 2. Further, a bus 7 is connected to the interface 12 outside the NAL device 2. Therefore, the controller 22 is connected to the CPU 5 via the interface 12 and the bus 7.

  A data processing unit 20 is connected to the interface 13 in the NAL device 2. Further, a bus 7 is connected to the interface 13 outside the NAL device 2. Therefore, the data processing unit 20 is connected to the CPU 5 via the interface 13 and the bus 7.

  A data buffer 23 is connected to the interface 14 in the NAL device 2. Further, the bus 14 is connected to the interface 14 outside the NAL device 2. Therefore, the data buffer 23 is connected to the CPU 5 via the interface 14 and the bus 7.

  FIG. 3 is a block diagram showing a specific configuration of the data processing unit 20 shown in FIG. As shown in FIG. 3, the data processing unit 20 includes interfaces 40 to 44, registers 30 to 34, a selection circuit 50, and processing units 51 and 52.

  A selection circuit 50 is connected to each output of the registers 30 to 33. A processing unit 51 is connected to the output of the selection circuit 50. A register 34 is connected to the output of the processing unit 51. A processing unit 52 is connected to the output of the register 34.

  A register 30 is connected to the interface 40 in the data processing unit 20. The interface 10 is connected to the interface 40 outside the data processing unit 20. Therefore, the register 30 is connected to the interface 10 via the interface 40.

  Registers 31 and 32 are connected to the interface 41 inside the data processing unit 20. Further, the interface 13 is connected to the interface 41 outside the data processing unit 20. Therefore, the registers 31 and 32 are connected to the interface 13 via the interface 41.

  A register 33 is connected to the interface 42 inside the data processing unit 20. In addition, the data buffer 23 is connected to the interface 42 outside the data processing unit 20. Therefore, the register 33 is connected to the data buffer 23 via the interface 42.

  A selection circuit 50 is connected to the interface 43 in the data processing unit 20. Further, the controller 22 is connected to the interface 43 outside the data processing unit 20. Therefore, the selection circuit 50 is connected to the controller 22 via the interface 43.

  A processing unit 52 is connected to the interface 44 inside the data processing unit 20. Further, the DMA buffer 21 is connected to the interface 44 outside the data processing unit 20. Therefore, the processing unit 52 is connected to the DMA buffer 21 via the interface 44.

  Hereinafter, the operation of the NAL device 2 according to the present embodiment will be described.

<Generation of NAL unit corresponding to non-image data>
The NALization apparatus 2 starts an AU delimiter (Access Unit Delimiter), SPS (Sequence Parameter Set), PPS (PPS) before starting to generate a NAL unit corresponding to image data (in the following description, it is assumed to be slice data). NAL units corresponding to various non-image data such as Picture Parameter Set) and SEI (Supplemental Enhancement Information) are generated.

  FIG. 4 is a diagram showing a configuration of the NAL unit 60 corresponding to non-image data. The NAL unit 60 has a configuration in which a NAL header 61, an RBSP 62 (RBSP: Raw Byte Sequence Payload) corresponding to non-image data, and a trailing bit 63 as necessary are arranged in this order. The trailing bit 63 is an adjustment bit for performing byte alignment processing. By adding the trailing bit 63 as necessary, the total bit length of the RBSP 62 and the trailing bit 63 is 8 bits. Aligned to an integer multiple of (1 byte). The processing for determining whether to add the trailing bit 63 and the addition processing are executed by the processing unit 51 shown in FIG.

  2 and 3, data relating to the NAL header 61 is stored in the register 32 from the bus 7 via the interfaces 13 and 41 under the control of the CPU 5. Data relating to the NAL header 61 is input from the register 32 to the selection circuit 50 as data S3.

  Data regarding the RBSP 62 is stored in the data buffer 23 from the bus 7 via the interface 14 under the control of the CPU 5. Further, data related to the RBSP 62 is read from the data buffer 23 under the control of the controller 22 and stored in the register 33 via the interface 42. Data relating to the RBSP 62 is input from the register 33 to the selection circuit 50 as data S4.

  The controller 22 inputs a selection signal S5 indicating that the data S3 and S4 should be selected in this order to the selection circuit 50 via the interface 43. As a result, data S6 in which data S3 (NAL header) and data S5 (RBSP) are arranged in this order is output from the selection circuit 50. Further, the processing unit 51 adds the trailing bit 63 to the data S6 as necessary, thereby generating the NAL unit 60 having the configuration shown in FIG. The NAL unit 60 output from the processing unit 51 is stored in the register 34.

  As the NAL unit 60 corresponding to the non-image data, a NAL unit 60A related to the AU delimiter, a NAL unit 60B related to the SPS, a NAL unit 60C related to the PPS, and a NAL unit 60D related to the SEI are generated and generated as necessary. They are stored in the register 34 in the order shown (see FIG. 7).

  Although not shown in FIG. 4, it is also possible to insert a predetermined start code at the head of each NAL unit 60 as necessary. Referring to FIGS. 2 and 3, data relating to the start code is stored in register 31 from bus 7 through interfaces 13 and 41 under the control of CPU 5. Data relating to the start code is input from the register 31 to the selection circuit 50 as data S2. By causing the selection circuit 50 to select the data S2 before the data S3, the data S2 (start code), the data S3 (NAL header), the data S5 (RBSP), and the trailing bit are arranged in this order. Unit 60 can be generated.

<Generation of NAL unit corresponding to slice data>
When the generation of the NAL unit 60 corresponding to the non-image data is completed, the NAL device 2 next performs the generation process of the NAL unit corresponding to the slice data. The start timing of this processing is notified from the CPU 5 to the controller 22 via the bus 7 and the interface 12.

  FIG. 5 is a diagram illustrating a configuration of the NAL unit 70 corresponding to slice data. Similar to the NAL unit 60 shown in FIG. 4, the NAL unit 70 has a configuration in which a NAL header 71, an RBSP 72 corresponding to slice data, and a trailing bit 73 as necessary are arranged in this order. . The trailing bit 73 is an adjustment bit for performing byte alignment processing. By adding the trailing bit 73 as necessary, the total bit length of the RBSP 72 and the trailing bit 73 is 8 bits. Aligned to an integer multiple of (1 byte). The processing for determining whether to add the trailing bit 73 and the addition processing are executed by the processing unit 51 shown in FIG.

  2 and 3, data related to the NAL header 71 is stored in the register 32 from the bus 7 via the interfaces 13 and 41 under the control of the CPU 5. Data relating to the NAL header 71 is input from the register 32 to the selection circuit 50 as data S3.

  Data regarding the RBSP 72 is stored in the register 30 from the encoding device 1 via the interfaces 10 and 40. Alternatively, data relating to the RBSP 72 is written from the encoding device 1 to the memory 4 via the bus 6 and then stored in the register 30 from the memory 4 via the bus 6 and the interfaces 10 and 40. Data relating to the RBSP 72 is input from the register 30 to the selection circuit 50 as data S1.

  The controller 22 inputs a selection signal S5 indicating that the data S3 and S1 should be selected in this order to the selection circuit 50 via the interface 43. As a result, data S6 in which data S3 (NAL header) and data S1 (RBSP) are arranged in this order is output from the selection circuit 50. Further, the processing unit 51 adds the trailing bit 73 to the data S6 as necessary, thereby generating the NAL unit 70 having the configuration shown in FIG. The NAL unit 70 output from the processing unit 51 is stored in the register 34.

  At this time, since the NAL units 60A to 60D corresponding to the non-image data are already stored in the register 34, the NAL unit 70 corresponding to the slice data is stored in the register 34 after the NAL unit 60D. (See FIG. 7).

  Although not shown in FIG. 5, it is possible to insert a predetermined start code at the head of the NAL unit 70 as necessary. The start code insertion method is the same as that described above.

<Formation of filler NAL>
In the NAL device 2 according to the present embodiment, after generating the NAL unit 70 corresponding to the slice data, it is possible to generate a filler NAL as necessary. The start timing of this processing is notified from the CPU 5 to the controller 22 via the bus 7 and the interface 12.

  FIG. 6 is a diagram illustrating the configuration of the filler NAL 80. As with the NAL unit 60 shown in FIG. 4, the filler NAL 80 has a configuration in which a NAL header 81, an RBSP 82 corresponding to the filler NAL, and a trailing bit 83 as necessary are arranged in this order. The trailing bit 83 is an adjustment bit for performing byte alignment processing. By adding the trailing bit 83 as necessary, the total bit length of the RBSP 82 and the trailing bit 83 is 8 bits. Aligned to an integer multiple of (1 byte). The processing for determining whether to add the trailing bit 83 and the addition processing are executed by the processing unit 51 shown in FIG.

  Referring to FIGS. 2 and 3, data related to NAL header 81 is stored in register 32 from bus 7 via interfaces 13 and 41 under the control of CPU 5. Data relating to the NAL header 81 is input from the register 32 to the selection circuit 50 as data S3.

  As data regarding the RBSP 82, predetermined dummy data (for example, 0xFF) is stored in the data buffer 23 under the control of the CPU 5. Here, assuming that the data capacity of the data buffer 23 is one page, the RBSP 82 of the filler NAL 80 normally has a data size of N pages (N is a natural number).

  Under the control of the controller 22, the dummy data is read from the data buffer 23 and stored in the register 33 via the interface 42. The data is input from the register 33 to the selection circuit 50 as data S4.

  The controller 22 inputs a selection signal S5 indicating that the data S3 and S4 should be selected in this order to the selection circuit 50 via the interface 43. At this time, regarding the data S4, the dummy data is read from the data buffer 23 to the register 33 and the data S4 from the register 33 to the selection circuit 50 according to the data size of the RBSP 82 notified from the CPU 5 to the controller 22. Input is executed repeatedly.

  As a result, the selection circuit 50 outputs data S6 in which data S3 (NAL header) and N pages of data S4 (RBSP) are arranged in this order. Further, the processing unit 51 adds the trailing bit 83 to the data S6 as necessary, thereby generating the filler NAL 80 having the configuration shown in FIG. The filler NAL 80 output from the processing unit 51 is stored in the register 34.

  At this time, since the NAL units 60A to 60D corresponding to the non-image data and the NAL unit 70 corresponding to the slice data are already stored in the register 34, the filler NAL 80 is stored in the register 34 after the NAL unit 70. It is stored (see FIG. 7).

  Although not shown in FIG. 6, a predetermined start code can be inserted at the head of the filler NAL 80 as necessary. The start code insertion method is the same as that described above.

  Further, after the filler NAL 80 is stored in the register 34, the NAL unit 60E related to EOS (End Of Sequence) and the EOS (End Of Stream) are processed in the same manner as the generation of the NAL unit 60 corresponding to the non-image data described above. The NAL unit 60 </ b> F is generated as necessary, and is stored in the register 34 after the filler NAL 80.

<Generation of access unit>
FIG. 7 is a diagram showing the contents stored in the register 34. FIG. 8 is a diagram showing the configuration of the access unit 90.

  Referring to FIG. 7, according to the operation so far, the register 34 stores the NAL unit 60A related to the AU delimiter, the NAL unit 60B related to SPS, the NAL unit 60C related to PPS, the NAL unit 60D related to SEI, and slice data (main picture) The corresponding NAL unit 70, filler NAL 80, NAL unit 60E related to EOS, and NAL unit 60F related to EOS are stored in this order.

  By outputting these NAL units from the register 34 in the order stored in the register 34 (that is, from the oldest stored time), as shown in FIG. 8, an access unit 90 with the AU delimiter at the head is obtained. .

  Referring to FIG. 3, access unit 90 is output from register 34 as data S <b> 7 and input to processing unit 52. The processing unit 52 checks whether or not the same bit pattern as the start code bit pattern exists in the access unit 90, and if it exists, inserts a predetermined bit string into the bit pattern. . Thereby, the generation of the pseudo start code in the access unit 90 is prevented. As a modification, the processing unit 52 is not inside the data processing unit 20 but outside the data processing unit 20, for example, between the interface 14 and the data buffer 23, or between the interface 10 and the data processing unit 20. It is also possible to do.

  The access unit 90 is output from the processing unit 52 as data S8 and is stored in the DMA buffer 21 via the interface 44. Then, it is stored in the memory 4 from the DMA buffer 21 via the interface 11 and the bus 6. The transfer of the access unit 90 from the DMA buffer 21 to the memory 4 is started when the capacity of the DMA buffer 21 becomes full under the control of the controller 22. The controller 22 can stop the NAL processing by the data processing unit 20 while transferring the access unit 90 to the memory 4. Meanwhile, since it is necessary to stop the input of slice data to the register 30, the controller 22 outputs a Wait signal to the preceding encoding device 1.

  When the output of the access unit 90 from the NAL device 2 is completed, the controller 22 notifies the CPU 5 of the completion via the interface 12 and the bus 7. This notification is executed as an interrupt process to the CPU 5. In response to this, the CPU 5 sends a PES processing start instruction to the PES device 3 subsequent to the NAL device 2.

  As a modification, the DMA buffer 21 and the register 34 (see FIG. 3) may be combined. As another modification, the DMA buffer 21 is omitted, and the access unit 90 output from the data processing unit 20 is directly connected to the subsequent PES device 3 via the interface 11 (that is, via the bus 6 and the memory 4). (Without)

<Summary>
As described above, according to the NAL device 2 according to the present embodiment, since the NAL device 2 (data processing device) is configured by hardware, the memory bus bandwidth is not expanded and the operating frequency is not increased. Thus, it is possible to realize high speed NAL processing. Moreover, the non-image data payload input processing and NAL header input processing are executed by software processing under the control of the CPU 5, thereby simplifying the device configuration as compared with the case where all processing is realized by a hardware configuration. Can be achieved.

  Further, according to the NAL device 2 according to the present embodiment, the generation processing of the access unit 90 by arranging a plurality of NAL units in a predetermined order can be simplified by cooperation of the hardware configuration and the software processing. It can be realized.

  Furthermore, according to the NAL device 2 according to the present embodiment, the filler NAL 80 generation process can be easily realized by the cooperation of the hardware configuration and the software process. In addition, by repeatedly reading the dummy data from the data buffer 23, it is possible to generate a filler NAL 80 having a capacity larger than the capacity of the data buffer 23.

H. 1 is a block diagram illustrating a part of a configuration of an image processing system compliant with the H.264 standard. FIG. 2 is a block diagram showing a specific configuration of the NAL device shown in FIG. 1. FIG. 3 is a block diagram illustrating a specific configuration of a data processing unit illustrated in FIG. 2. It is a figure which shows the structure of the NAL unit corresponding to non-image data. It is a figure which shows the structure of the NAL unit corresponding to slice data. It is a figure which shows the structure of filler NAL. It is a figure which shows the storage content of a register. It is a figure which shows the structure of an access unit.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Encoding apparatus 2 NAL conversion apparatus 3 PES conversion apparatus 4 Memory 5 CPU
6, 7 Bus 20 Data processing unit 21 DMA buffer 22 Controller 23 Data buffer 30-34 Register 50 Selection circuit 60, 70 NAL unit 61, 71, 81 NAL header 62, 72, 82 RBSP
80 Filler NAL
90 Access unit

Claims (3)

A data processing unit capable of inputting a payload of image data encoded by an external encoding device from the encoding device;
A non-image data payload can be input via an external bus under the control of an external control device, and includes a data buffer capable of storing the input non-image data payload.
A predetermined NAL (Network Abstraction Layer) header can be input to the data processing unit via the external bus under the control of the control device,
The data processing unit
By adding the NAL header to the payload of the non-image data read from the data buffer, a first NAL unit corresponding to the non-image data is generated,
By adding the NAL header to the payload of the image data, a second NAL unit corresponding to the image data is generated,
A data processing apparatus that generates an access unit by arranging the first NAL unit and the second NAL unit. The data processing unit has a register capable of storing a plurality of NAL units,
The data processing unit stores the generated first NAL unit and the second NAL unit in the register in this order, and the first NAL unit and the second NAL unit in this order from the register. The data processing apparatus according to claim 1, wherein the access unit is generated by outputting. The data buffer can store predetermined dummy data,
The data processing device according to claim 1, wherein the data processing unit can generate a filler NAL by repeatedly reading the dummy data from the data buffer.

Images