JP2005341567A - Parser for h.264 video decoder - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a parser for H.264 video decoders. <P>SOLUTION: The parser comprises a parsing means for parsing encoded bitstreams input from outside, an EG (Exponential Golomb) decoder for decoding environmental information and slice headers in the bitstream data parsed by the parsing means, and an internal memory for storing the slice headers decoded by the EG decoder. In the parser, the internal memory has a plurality of storage areas, each storing the slice headers. The decoded slice headers are sequentially stored in the internal memory, by switching. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to H.264. The present invention relates to a parser of a H.264 video decoder (Video Decoder), and more particularly to H.264. The present invention relates to an effective syntax analyzer structure for video codec syntax analysis according to the H.264 standard.

  H. H.264 is a package based video that can be applied to video conferencing and broadcasting, streaming services, etc., as a proposal for a video codec proposed by the International Telecommunications Union (ITU-T). One of the compression techniques. H. H.264 is H.264. MPEG-2 based on H.262, or H.264 Compared with MPEG-4 based on H.263, the compression rate is further improved.

  In general, one video is composed of a plurality of consecutive pictures, and a picture displayed on one screen is composed of a plurality of slices. Each slice is again composed of a plurality of macroblocks (Macro Blocks) including a plurality of pixels (Pixels), and each slice includes a slice header (Slice Header) indicating information on the corresponding slice. On the other hand, H. Data compressed using the H.264 compression technique is stored in a predetermined buffer. Then, the bit stream (Bitstream) of the data stored in the buffer by the syntax analyzer is analyzed, and the analyzed data is an entropy decoder, an inverse transformer, a predictor, and a deblock. It is processed by a hardware module (Hardware Module) such as a king filter (Deblocking Filter) and displayed on the screen in units of pictures. The syntax analyzer is responsible for reading the bitstream from the buffer, analyzing it, and processing it for use by other hardware modules as described above.

  On the other hand, the bit stream read from the buffer by the syntax analyzer includes slice data, slice header, and various H.264 data constituting the current picture. H.264 environment information is stored. H. The H.264 environment information includes a large number of sequence parameter sets (hereinafter referred to as SPS: Sequence Parameter Set) and picture parameter cells (hereinafter referred to as PPS: Picture Parameter Set), and the SPS is used to construct one complete video sequence. Parameters include parameters such as own ID and maximum number of frames, picture procedure, width and height of decoded picture. In addition, the PPS selects the ID of the ID and the selected SPS ID, CAVLC (Context Adaptive Variable Length Coding) or CABAC (Context-based Adaptive Binary Coding) entropy coding to select and slice parameters. And parameters for prediction, quantization, and deblocking.

  On the other hand, H. In H.264, not only the actual video data but also the above-described environment information and a part of the slice header are compressed in the form of an EG code (Exp-Golomb code). Therefore, H.H. In H.264, for smoother syntax analysis, a process and an apparatus for decoding them quickly and accurately are essential. In addition, H.C. H.264 has a more complicated syntax structure than other video codecs in order to increase the compression rate. Such H. The complexity of H.264 compression technology makes actual implementation very difficult.

  The object of the present invention is to provide a fast and accurate H.264. It is an object of the present invention to provide an effective structured syntax analyzer capable of performing H.264 syntax analysis.

  According to one aspect of the present invention for achieving the above-described object, The H.264 data processing method is an H.264 process. In order to simultaneously store a plurality of slice headers respectively associated with H.264 slice data, a plurality of areas in the internal memory are stored in H.264. Assigning to H.264 parsing circuitry.

  In a preferred embodiment, the method includes transmitting a plurality of parameter sets associated with the plurality of slice headers from an external memory to the internal memory.

  In this embodiment, the transmitting step includes transmitting a plurality of first parameter sets to a first allocation area of the internal memory in response to a first slice header analysis with reference to the first parameter set; Transmitting a plurality of second parameter sets to a second allocation region of the internal memory in response to a second slice header analysis with reference to the set.

  In this embodiment, the plurality of first parameter sets is a first H.P. H.264 PPS and 1H. H.264 SPS, and the plurality of second parameter sets are second H.264. H.264 PPS and 2H. H.264 SPS.

  In this embodiment, the method provides the first PPS and the first SPS to a first sppage of a video processing pipeline to process a first slice associated with a first slice header of the plurality of slice headers. The method further includes the step of:

  In this embodiment, the first stage provides an output based on the processing of the first slice, and the method processes a second slice associated with a second slice header of the plurality of slice headers. Providing the second PPS and the second SPS to the first stage in order to process the output from the first stage in parallel with the first PPS and the first SPS in the second of the video processing pipeline. Providing to the stage.

  In this embodiment, the method further includes transmitting the first PPS and the first SPS to a second region of the internal memory before providing the first PPS and the first SPS to the first stage in parallel. .

  In this embodiment, the method includes changing a pointer associated with a second stage of the video processing pipeline to the first region before providing the first PPS and the first SPS to the second stage in parallel. Further included.

  In accordance with another aspect of the present invention, H.264 processing circuit H.264 slice header and non-VCL, HAL to detect NAL units. A syntactic analysis circuit configured to analyze the H.264 bit stream, coupled to the syntactic analysis circuit, And a memory allocated with a plurality of areas for storing 264 slice headers in parallel.

  In a preferred embodiment, the memory includes an internal memory contained within the same integrated circuit package as the syntax analysis circuit.

  In a preferred embodiment, the circuit is coupled to the syntax analysis circuit and the memory, and is assigned the plurality of first parameter sets in response to receiving a first slice header analyzed with reference to a first parameter set. A plurality of second parameter sets are transmitted to the first region of the plurality of regions and the second parameter set is received in response to receiving the second slice header analyzed with reference to the second parameter set. And a control circuit for transmitting to the second region.

  According to another aspect of the invention, A computer program product for H.264 application data processing comprises a computer readable medium having a computer readable program code embedded therein and an H.264 computer program product. In a computer program product that processes H.264 data, H.264 syntax analysis circuit is processed each time. Computer readable program code that allocates a plurality of regions in internal memory for storing a plurality of slice headers associated with H.264 slice data in parallel.

  The parser of the present invention allows real-time processing of data with reduced delay time that can be generated as it progresses from the current slice to the next slice. In addition, the syntax analyzer of the present invention not only can minimize the processor load of the syntax analyzer by utilizing a decoder included in an external hardware module, but the syntax analyzer of the present invention utilizes a small internal memory. Thus, the size of the entire syntax analyzer can be reduced.

  FIG. 1 shows an H.264 including a syntax analyzer 104 according to a preferred embodiment of the present invention. 2 is a block diagram showing a H.264 application processing circuit. FIG. Referring to FIG. An input buffer 102 of the H.264 processing circuit 100, a syntax analysis circuit 104, a control circuit 106, an internal memory 108, and an EG decoder 109 are included. H. The H.264 processing circuit 100 is connected to the external memory 120, the video data processing device or pipeline 140, and the external buffer 110.

  In the preferred embodiment of the present invention, the internal memory 108 is a static random access memory (SRAM) and is included in the same integrated circuit package as the parser 104. In another embodiment of the present invention, the input buffer 102 is a 32-bit double buffer that is controlled by the control circuit 106 and stores a bit stream called from the external buffer 110. In another embodiment of the present invention, the external buffer 110 is an H.264 buffer for storing coded picture information accessible as a bitstream. It is a H.264 coded picture buffer (CPB). In the present specification, “external” means a circuit different from the same integrated circuit package as the syntax analyzer 104.

  The syntax analysis circuit 104 reads the data bit stream from the input buffer 102 and analyzes the start code SC indicating the start of each slice header in the bit stream. In particular, the start code is accompanied by a slice header including information indicating how the sub-sequence slice data is processed. In an embodiment of the present invention, the information indicates how the data to be processed can be provided directly or indirectly with reference to a parameter not included in the slice header.

  The syntax analysis circuit 104 can store slice header information in the internal memory 108 via the control circuit 106, and externally transmits the next slice data of the slice header (that is, data including a VCL (video coding layer) NAL unit). Communicate to video data processing unit / pipeline 140. The parsing circuit 104 analyzes the bitstream to confirm the non-VCL NAL unit, and the non-VCL NAL unit is transmitted to the control circuit 106. The non-VCL NAL unit can be data indicating whether related slice data is processed by the video data processing device / pipeline 140, such as SPS (Sequence Parameter Sets) and PPS (Picture Parameter Sets).

  The control circuit 106 decodes the non-VCL NAL unit using the EG decoder 109 and stores the result in the external memory 120. For example, a non-VCL NAL unit including PPS and SPS information can be decoded by the Exp-Glomb decoder 109 and stored in the external memory 120. On the other hand, in another embodiment of the present invention, an EG decoder included in the external video data processing device / pipeline 104 can be used.

  On the other hand, H. In H.264, in order to distinguish video data having the same bit configuration as the start code SC from the video data from the start code, an emulation bit (EB: Emulation Bit) having a specific value is inserted into the corresponding video data, and syntax analysis is performed. The circuit 104 removes this emulation bit EB from the NAL unit.

  The control circuit 106 decodes the environment information using the EG decoder 109, stores the result value in the external memory 120, and loads the appropriate information into the internal memory 108 according to the slice header to load the video data processing device 140. The other hardware modules included in can be used. On the other hand, as described above, H.P. In H.264, a part of the slice header is also compressed in the form of an EG code. Such a slice header is also decoded through the EG decoder 109 and stored in the internal memory 108. On the other hand, the EG decoder 109 can be realized separately in the syntax analyzer 100 as shown in FIG. 1, but is included in the external video data processing device 140 in order to minimize the load on the syntax analyzer 100. An EG decoder of an entropy decoder can also be used.

  FIG. 2 is a block diagram illustrating an operation of the syntax analyzer 100 that performs decoding using the EG decoder 204 included in the outer entropy decoder 202. As shown in FIG. 2, the entropy decoder 202 included in the external video data processing apparatus 130 is used for decoding the data ue (V) and se (V) which are largely compressed in the form of Exo-Golomb code. An EG decoder 204 and a CABAC (Context-Adaptive Binary Arithmetic Coding) block 206 and a CAVLC (Context-Adaptive Variable Length Coding) block 206 for processing video data are included.

  On the other hand, as described above, the general H.264 standard. H.264 video data processing apparatus includes H.264, such as an entropy decoder, an inverse discrete cosine transformer, a predictor, and a deblocking filter. H.264 hardware modules, each such hardware module processes data in units of slices for one picture and stores it in the picture buffer, and if all the slices constituting one picture are processed Display on the screen. Then, each hardware module included in the video data processing apparatus 130 performs data processing with reference to the corresponding slice header information stored in the internal memory 108 in the process of processing the slice data. That is, the data processing of the corresponding slice cannot be executed before the syntax analysis of the slice header is completed. The corresponding slice header information must remain in the internal memory 108 until data processing by each hardware block is completed for one slice. Therefore, if a storage area is allocated to the internal memory 108 for only one slice header, the next slice is not in a standby state until all hardware modules have been processed for the slice already in progress. In other words, a delay for that occurs. On the other hand, one picture consists of a plurality of slices, and if such a delay is accumulated, real-time data processing may become impossible.

  Thus, in embodiments of the present invention, multiple regions of internal memory are allocated to parallel storage and access of multiple slice headers while associated slices are processed. A plurality of areas of the internal memory can store PPS and SPS information related to slice data corresponding to the slice header in parallel.

  In the example shown in FIG. 3, two slice header storage areas 304 and 306 are placed in the internal memory 108, and the slice headers SH <b> 1 and SH <b> 2 that have been analyzed are stored in parallel in the two storage areas 304 and 306. SH1 and SH2 correspond to the two slice data SD1 and SD2, respectively. Slice headers stored in parallel can be approached for use in processing each slice data in other pipeline stages.

  Referring to FIG. 3, if a bitstream 302 including two slice information is input, the syntax analyzer 300 first detects a start code SC of the first slice from the bitstream and performs a search for the first slice. Perform syntax analysis on a NAL basis. The syntax analyzer 300 decodes the slice header information SH1 for the first slice through the EG decoder and stores it in the first storage area 304 of the internal memory 108. Then, the first slice data SD1 input thereafter is input to the P1 module of the video data processing apparatus, that is, the entropy decoder. Also, the syntax analyzer 300 decodes the slice header information SH2 for the second slice continuously input from the bit stream 302 through the EG decoder and stores the decoded information in the second storage area 306 of the internal memory 108. Then, the second slice data SD2 is continuously input to the entropy decoder P1. On the other hand, the entropy decoder P1 processes the first slice data SD1 with reference to the first slice header SH1 stored in the first storage area 304 of the internal memory 108, and processes it on the next module (P2 module) of the video data processing device. ). When the second slice data SD2 is input from the syntax analyzer 300, the second slice data SD2 is processed with reference to the second slice header SH2 stored in the second storage area 306 of the internal memory 108. Thus, in the present invention, two storage areas are allocated to the internal memory and each slice header information is stored. Through this, the present invention allows real-time processing of data by reducing the delay time that can be generated to progress from the current slice to the next slice.

  On the other hand, in order to process one slice data, the PPS and SPS environment information of the corresponding slice must be referred to, and such PPS and SPS environment information is included in the bit stream from the CPB as described above. It is input while The syntax analyzer 100 decodes such environment information SPS and PPS via the EG decoder 109 and stores them in the external memory 120. (Of course, such PPS and SPS environment information is not always stored in the external memory only through the above-described process. In order to further improve the data processing speed, PPS and SPS environment information is defined in advance. On the other hand, when slice data is processed by the external hardware module, the corresponding PPS is read from the external memory 120 with reference to the PPS ID of the corresponding slice included in the slice header. Information is called and stored in the internal memory 108 for use. The corresponding SPS information is called from the external memory 120 with reference to the SPS ID included in the PPS information and stored in the internal memory 108. For example, as shown in FIG. 4, if the corresponding PPS ID of the current slice included in the slice header 2SH2 is 5, the environment information of PPS5 is called from the external memory 120 and stored in the internal memory 108. If the SPS ID included in the PPS 5 is 3, the environment information of the SPS 3 is called from the external memory 120 and stored in the internal memory 108.

  FIG. 5 is a diagram illustrating an embodiment of parameters stored in an internal memory having two storage areas according to the present invention. Each of the parameters stored in the internal memory 108 after the process described with reference to FIG. 4 is performed. Indicates the parameter set. Referring to FIGS. 4 and 5, the first storage area 304 of the internal memory 108 stores a previous parameter set, that is, slice header information SH1 and PPS information PPS4, SPS information SPS6, and previous video for the previous slice. Parameters are stored. In the second area 306 of the internal memory, a current parameter set, that is, slice header information SH2 and PPS information PPS5 and SPS information SPS3 of the current slice, and parameters for processing the current video are stored. When the processing of the previous slice is completed by each hardware module, the parameter set of the next slice is stored in the first storage area 304.

  2, 3, and 4 according to the embodiment of the present invention, the syntax analysis circuit 104 decodes the PPS and SPS using the EG decoder 109 and stores the PPS and SPS information in the external memory 120. To do. In order to control the processing of SD1 by P1, slice data SD1 is provided to P1, and parameters from PPS4 and SPS6 are also provided to P1. The output from P1 based on the processing of SD1 is provided to P2 along with the parameters from PPS4 and SPS6. In parallel to control the processing of SD2 by P1, SD2 is provided to P1, and parameters from PPS5 and SPS3 are provided to P1. The output from P1 based on the processing of SD2 is provided to P2 along with the parameters from PPS5 and SPS3. That is, the PPS and SPS parameters are shifted to control the processing of each slice data in successive stages of the pipeline.

H. proposed by the present invention. It is a block diagram for demonstrating operation | movement of a H.264 syntax analyzer. The H.264 of the present invention performs decoding using an EG decoder included in an external entropy decoder. FIG. 6 is a block diagram illustrating the operation of an H.264 syntax analyzer. H. of the present invention. 6 is a diagram illustrating an operation of switching and storing a slice header in an internal memory by an H.264 syntax analyzer. FIG. In the present invention, the corresponding H.D. It is a figure which shows the operation | movement which reads H.264 environment information. FIG. 5 is a diagram illustrating an embodiment in which environment information read through the process of FIG. 4 is stored in each area of an internal memory.

Claims (24)

  H. processed In order to simultaneously store a plurality of slice headers respectively associated with H.264 slice data, a plurality of areas in the internal memory are stored in H.264. H.264 syntax analysis circuit includes a step of assigning to H.264 syntax analysis circuit. H.264 data processing method.   The method of claim 1, comprising transmitting a plurality of parameter sets associated with the plurality of slice headers from an external memory to the internal memory. H.264 data processing method. The transmission stage includes
Transmitting a plurality of first parameter sets to a first allocation region of the internal memory in response to a first slice header analysis with reference to the first parameter set;
The method of claim 2, further comprising: transmitting a plurality of second parameter sets to a second allocation region of the internal memory in response to a second slice header analysis with reference to the second parameter set. . H.264 data processing method.   The plurality of first parameter sets includes a first H.P. H.264 PPS and 1H. H.264 SPS, and the plurality of second parameter sets are second H.264. H.264 PPS and 2H. The H.264 according to claim 3, wherein the H.264 SPS is H.264 SPS. H.264 data processing method.   Providing the first PPS and the first SPS to a first sppage of a video processing pipeline for processing a first slice associated with a first slice header of the plurality of slice headers; H. of claim 4. H.264 data processing method. The first stage provides an output based on processing of the first slice;
Providing the second PPS and the second SPS to the first stage to process a second slice associated with a second slice header of the plurality of slice headers;
6. The method of claim 5, further comprising: providing the first PPS and the first SPS to a second stage of the video processing pipeline in parallel to process the output from the first stage. H. H.264 data processing method.   7. The method of claim 6, further comprising transmitting the first PPS and the first SPS to the second area of the internal memory before providing the first PPS and the first SPS to the first stage in parallel. H. described. H.264 data processing method.   The method further comprises changing a pointer associated with the second stage of the video processing pipeline to the first region before providing the first PPS and the first SPS to the second stage in parallel. Item 6. The H.3. H.264 processing method. H. H.264 slice header and non-VCL, HAL to detect NAL units. A parser circuit configured to analyze the H.264 bitstream;
Connected to the syntax analysis circuit; And a memory allocated with a plurality of areas for storing H.264 slice headers in parallel. H.264 processing circuit.   10. The H.264 of claim 9, wherein the memory includes an internal memory included in the same integrated circuit package as the syntax analysis circuit. H.264 processing circuit.   A plurality of first parameter sets are coupled to the syntax analysis circuit and the memory and in response to receiving a first slice header analyzed with reference to a first parameter set, a first of the plurality of allocated areas. A control circuit for transmitting a plurality of second parameter sets to a second region of the allocated regions in response to receiving a second slice header analyzed with reference to the second parameter set The H.M. of claim 9 further comprising: H.264 processing circuit.   The plurality of first parameter sets includes a first H.P. H.264 PPS and 1H. H.264 SPS, and the plurality of second parameter sets are second H.264. H.264 PPS and 2H. The H.264 according to claim 11, wherein the H.264 SPS is H.264 SPS. H.264 data processing circuit. The circuit is
The method of claim 11, wherein the first PPS and the first SPS are provided to a first stage of a video processing pipeline for processing a first slice associated with a first header of the plurality of slice headers. H. described. H.264 data processing circuit. The first stage provides an output based on processing of the first slice;
The circuit is
Providing the second PPS and the second SPS to the first stage to process a second slice associated with a second slice header of the plurality of slice headers and processing an output from the first stage; 14. The H.264 of claim 13, wherein the first PPS and the first SPS are provided in parallel to the second stage of the video processing pipeline. H.264 data processing circuit. The control circuit includes:
15. The apparatus according to claim 14, wherein the first PPS and the first SPS are transmitted to the second area of the internal memory before the first PPS and the first SPS are transmitted to the second stage in parallel. Described in H. H.264 data processing circuit. The control circuit includes:
15. The H.D. of claim 14 wherein a pointer associated with a second stage of a video processing pipeline is changed to the first region before providing the first PPS and the first SPS to the second stage in parallel. H.264 data processing circuit. A computer-readable medium having a computer-readable program code embedded therein; In a computer program product that processes H.264 data,
H. H.264 syntax analysis circuit is processed each time. A computer program product comprising computer readable program code for allocating a plurality of areas in an internal memory for storing a plurality of slice headers associated with H.264 slice data in parallel.   The computer program product of claim 17, further comprising computer readable program code configured to communicate a plurality of parameter sets associated with a plurality of slice heads from an external memory to the internal memory. The program code configured to communicate is
The first slice header;
Computer readable program code for communicating a plurality of first parameter sets to a first assigned region of the internal memory in response to analyzing the first slice header with reference to the first parameter set;
Computer readable program code for communicating a plurality of second parameter sets to a second assigned region of the internal memory in response to analyzing the second slice header with reference to the second parameter set. The computer program product of claim 18, wherein the computer program product is a computer program product.   The plurality of first parameter sets includes a first H.P. H.264 PPS and 1H. H.264 SPS, and the plurality of second parameter sets are second H.264. H.264 PPS and 2H. The computer program product of claim 19, wherein the computer program product is H.264 SPS.   A computer readable program configured to provide the first PPS and the first SPS to a first stage of a video processing pipeline for processing a first slice associated with a first header of the plurality of slice headers. The computer program product of claim 20, further comprising code. The first stage provides an output based on processing of the first slice;
Computer readable program code configured to provide the second PPS and the second SPS to the first stage to provide processing of a second slice associated with a second slice header of the plurality of slice heads When,
And further comprising computer readable program code configured to provide the first PPS and the first SPS to a second stage of a video processing pipeline in parallel to provide processing of the output from the first stage. 22. A computer program product according to claim 21.   Computer readable program code configured to communicate the first PPS and the first SPS to the second area of the internal memory before providing the first PPS and the first SPS to the second stage in parallel. The computer program product of claim 22, comprising:   Computer readable program code configured to change a pointer associated with a second stage of the video processing pipeline to the first region before providing the first PPS and the first SPS to the second stage in parallel The computer program product of claim 22 further comprising:

Images