JP4731343B2 - Decoding device - Google Patents

Description

  The present invention relates to a decoding apparatus using a moving picture compression technique such as MPEG (Moving Picture Experts Group) -4 AVC / H.264.

  In a conventional moving image encoding apparatus, in encoding such as ISO / IEC14496-2 (MPEG-4), an image is encoded in units of one picture, and a generated bitstream has a synchronization code at the head of a picture. It was attached. For example, in MPEG-4, there is a layer called a VOP header that always starts with a synchronization code (0x000001B6) called VOP Start Code at the start of a picture, and subsequently data of each macroblock is encoded. Therefore, even if the decoding device is out of synchronization due to an environment where errors are mixed and decoding is impossible, it is determined that the picture information starts by detecting the above VOP Start Code, and synchronization is always performed from the next picture. It is a mechanism that can be removed.

  However, in the case of an encoding method such as ISO / IEC14496-10 (MPEG-4 AVC / H.264) defined in recent years, picture data, parameter sets, etc. are all incorporated into the same layer called Network Abstraction Layer (NAL). Is transmitted. For this reason, since all data has the same start code, it can be interpreted whether it is picture data or a parameter by analyzing its contents in detail (see, for example, Non-Patent Document 1).

  Furthermore, there is no header or the like that explicitly indicates the picture boundary, and it is a mechanism for determining whether it is a picture boundary from the relationship between the current NAL and the parameters included in the previous NAL. For example, the encoded data of a picture itself is divided into units called slices and transmitted as slice NALs. If multiple slices NAL are received in succession, whether they are slices of the same picture or include picture boundaries. Judgment is made based on whether the frame_num number in the header of the slice NAL is different. Alternatively, a special NAL (access_Unit_delimiter_NAL) that is inserted only at the picture boundary is also defined, and depending on the application, the picture boundary can be explicitly indicated by using this, but this is not essential, so as a result Judgment is based on the value of frame_num.

However, when an error is mixed in such a stream, the frame_num in the slice header is a fixed-length code, so it is very difficult to distinguish the error, and the picture has changed when the value of the frame_num has changed. It cannot be determined whether the frame_num has changed due to the error or the frame_num has changed due to an error. This causes a problem that picture boundaries cannot be correctly determined. This problem also occurs when picture boundaries are performed based on access_Unit_delimiter_NAL.
"H.264 / AVC textbook" supervised by Satoshi Okubo, Impress Communications Inc., July 30, 2004.

The conventional decoding apparatus has a problem that the picture boundary cannot be correctly determined when an error is mixed in a slice header or access_Unit_delimiter_NAL used for determining the picture boundary.
The present invention has been made to solve the above problem, and an object of the present invention is to provide a decoding apparatus capable of correctly determining a picture boundary even when an error is mixed in a slice header or access_Unit_delimiter_NAL.

In order to achieve the above object, according to the present invention, each picture data is divided into a plurality of slice data, and a video stream in which the slice data is packetized and a multiplexed stream in which the audio stream is multiplexed are input. In the decoding device that decodes the video stream, the slice data is converted into the same original data based on the information included in the header of the packet that constitutes the video stream and the separation means that separates the video stream and the audio stream from the multiplexed stream. An AV separation unit comprising boundary information detection means for detecting boundary information for classifying each picture data, and the slice data for each picture data based on the boundary information notified from the boundary information detection means of the AV separation unit To decode and slice data into Comprising a issue unit, a video decoding unit, on the basis of a header of the slice data, further comprising, boundary information and identification information identification information detecting means for detecting identification information for classifying slice data for each same picture data Thus, the slice data is classified into the same picture data, and the slice data is decoded.

  According to the present invention, it is possible to provide a decoding device that can correctly determine a picture boundary even when an error is mixed in a slice header or access_Unit_delimiter_NAL.

Hereinafter, an embodiment of the present invention will be described with reference to the drawings.
FIG. 1 shows the configuration of a decoding apparatus according to the first embodiment of the present invention. The decoding device includes an A / V separation unit 10 and a video decoding unit 20.

  The A / V separation unit 10 receives a multiplexed stream obtained by multiplexing an audio stream and a video stream, separates the multiplexed stream into an audio stream and a video stream, and outputs the audio stream to an audio decoding unit (not shown). The stream is output to the video decoding unit 20. Here, a case will be described as an example where a TS multiplexed stream that is packetized as a TS (Transport Stream) based on ISO / IEC 14496-1 (MPEG-2 system) is used as the multiplexed stream.

  The A / V separator 10 includes a boundary detector 10a. The boundary detection unit 10a detects a picture boundary for classifying slice data for each original picture data based on the video stream obtained by the separation. Specifically, the process shown in FIG. 2 is performed for each TS packet. Hereinafter, a description will be given with reference to FIG.

  As described above, since the video stream is composed of TS multiplexed streams, first, the boundary detection unit 10a decodes the TS header, and when the TS packet forms the video stream, the payload_unit_start_indicator in the TS header is “ 1 ”is determined (step 2a).

  A TS packet whose payload_unit_start_indicator is “1” includes a PES (Packet Elementary Stream) header. Therefore, the boundary detection unit 10a further analyzes the stream, decodes the PES header, and extracts the PES configuration content (step 2b).

  When the boundary detection unit 10a detects a TS packet including a PES header, the boundary detection unit 10a determines whether the data_alignment_indicator of the PES header is “1” (step 2c).

  When data_alignment_indicator in the PES header is “1”, this indicates a picture boundary. Therefore, when the data_alignment_indicator is “1”, the boundary detection unit 10a outputs information indicating that the picture boundary has been detected to the video decoding unit 20 as boundary information (step 2d).

  The information indicating that the picture boundary has been detected may be, for example, an ID that is not used in the standard before sending the NAL of the video stream to the video decoding unit 20 in addition to connecting a signal line and setting the register. Insert a NAL or insert a pattern bit string that does not exist in the video stream. If the NAL type decoding unit 21 finds the NAL or special pattern, it is determined as a picture boundary, and at the same time, the information is discarded and notified. Is possible.

The video decoding unit 20 decodes the video stream based on the video stream and boundary information given from the A / V separation unit 10 to obtain data for each picture. The video decoding unit 20 includes a NAL type decoding unit 21, an additional information decoding unit 22, a slice header decoding unit 23, a picture boundary determination unit 24, a picture boundary processing unit 25, and a slice data decoding unit 26.
The NAL type decoding unit 21 determines whether the video stream supplied from the A / V separation unit is a NAL indicating additional information or a NAL composed of slices including video data, and the additional information is additional information. The decoding unit 22 and the slice data are output to the slice header decoding unit 23, respectively.

  The additional information decoding unit 22 decodes additional information other than video data. For example, video image size, bit rate, time information, or special information such as access_unit_delimiter_NAL.

  The slice header decoding unit 23 decodes header information (slice header) of each slice data in the video stream supplied from the A / V separation unit 10. This slice header includes parameters necessary for determining a picture boundary, such as frame_num. By sending these pieces of information to the picture boundary determining unit 24, it is determined whether or not this slice is information on a new picture. In addition, the picture boundary determination unit 24 determines that the stream input to the video decoding unit 20 corresponds to a picture boundary when the boundary information is given from the boundary detection unit 10a. These determination results are output to the picture boundary processing unit 25.

The picture boundary processing unit 25 performs processing associated with the picture boundary when the picture boundary determination unit 24 determines that it is a picture boundary.
Specifically, it is determined that the image data for the screen size obtained by the additional information decoding unit 22 has been decoded, and the content of the decoded picture buffer (DPB) for accumulating and displaying the decoded image is updated. Perform display processing if necessary. In addition, a parameter for each picture (for example, a macroblock number) is reset.

  The slice data decoding unit 26 decodes each slice data in the video stream input to the video decoding unit 20 in accordance with an instruction from the picture boundary processing unit 25 to obtain decoded data for each picture.

  As described above, in the decoding apparatus configured as described above, in addition to performing picture boundary determination based on information such as frame_num among the header information decoded by the slice header decoding unit 23, an A / V separation unit Also in the ten boundary detection units 10a, the picture boundary is determined based on the data_alignment_indicator of the PES header.

  In the decoding apparatus having the above configuration, the determination result of the picture boundary by the boundary detection unit 10a is given priority over the determination result of the picture boundary based on information such as frame_num of the slice header.

  Therefore, according to the decoding apparatus having the above configuration, even when an error is mixed in the frame_num of the slice header and the picture boundary cannot be detected, the boundary detection unit 10a of the A / V separation unit 10 detects the picture boundary. Therefore, the picture boundary can be correctly determined.

  Next explained is a decoding apparatus according to the second embodiment of the invention. FIG. 3 shows the configuration, and includes an A / V separation unit 30 and a video decoding unit 40. Here, an example in which TS (Transport Stream) packetized data based on ISO / IEC 14496-1 (MPEG-4 system) is used as the multiplexed stream input to the A / V separation unit 30 is described. Will be described.

  In addition, the multiplexed stream is obtained by multiplexing an audio stream and a video stream. Of these, video data is encoded by ISO / IEC 14496-10 (MPEG-4AVC / H.264). Assume that a NAL (Network Abstraction Layer) called access_Unit_delimiter_NAL that explicitly indicates a picture boundary as shown in FIG.

  The A / V separator 30 receives the multiplexed stream, separates it into an audio stream and a video stream, outputs the audio stream to an audio decoder (not shown), and outputs the video stream to the video decoder 40. Output.

  The A / V separation unit 30 includes a boundary detection unit 30a and a NAL control unit 30b. The boundary detection unit 30a detects a picture boundary for classifying the slice data for each original picture data based on the video stream obtained by the separation. The NAL control unit 30b controls access_Unit_delimiter_NAL according to the detection result of the picture boundary by the boundary detection unit 30a. Specifically, the boundary detection unit 30a and the NAL control unit 30b perform the process shown in FIG. 5 for each TS packet. Hereinafter, a description will be given with reference to FIG.

  As described above, since the video stream is composed of TS multiplexed streams, first, the boundary detection unit 30a decodes the TS header. If each TS packet constitutes the video stream, the ts_error_indicator in the TS header is “1”. "Is determined (step 5a).

An error is always included in a TS packet whose ts_error_indicator is “1”. When it is “0”, all the data in the TS packet can be trusted.
When ts_error_indicator is “0”, it is determined whether payload_unit_start_indicator in the TS header is “1” (step 5b).

  A TS packet whose payload_unit_start_indicator is “1” includes a PES (Packetized Elementary Stream) header. Therefore, the boundary detection unit 30a decodes the PES header (step 5c). When the boundary detection unit 30a detects the TS packet including the PES header, the boundary detection unit 30a determines whether the data_alignment_indicator included in the detected PES header is “1” (step 5d).

  When data_alignment_indicator of the PES header is “1”, it indicates a picture boundary. In this case, the boundary detection unit 30a outputs information indicating that the picture boundary has been detected to the picture boundary determination unit 44 of the video decoding unit 40 (step 5e).

  The information indicating that the picture boundary has been detected may be set in a register by connecting a signal line, for example, and is not used in the standard before outputting the NAL of the video stream to the video decoding unit 40. A NAL having an ID is inserted, or a bit string of a pattern that does not exist in the video stream is inserted. If the NAL type decoding unit 41 finds the NAL or special pattern, it is determined as a picture boundary, and this information is discarded at the same time. Notification is possible.

  When ts_error_indicator is “0”, since the video data does not contain an error, information indicating that the video data information is reliable is also output (step 5f).

  For information on whether or not an error is included in the video data, for example, a 1-bit flag “forbidden_zero_bit” in the NAL header of the video stream is rewritten to 1 indicating that there is an error, and the NAL type decoding unit 41 changes the NAL header. By checking when decoding, the video decoding unit 40 can be notified.

  On the other hand, when ts_error_indicator is “1” in step 5a, since payload_unit_start_indicator cannot be trusted, it is determined by directly analyzing the stream whether or not the PES header is included (step 5g).

  For example, the first 32 bits are analyzed, and it is determined whether or not packet_start_cod_prefix (24 bits) and stream_id (8 bits) are included.

  When it is determined that there is a PES header, the PES header is decoded to determine whether or not data_alignment_indicator included in the PES header is “1” (step 5h).

  When the data_alignment_indicator of the PES header is “1”, this indicates a picture boundary. In this case, the boundary detection unit 30a outputs information indicating that the picture boundary has been detected to the NAL control unit 30b (step 5i).

  The NAL control unit 30b determines whether the access_unit_delimiter_NAL is present in the stream without error following the PES header (step 5j).

  For example, the first 6 bytes may be analyzed to determine whether the information is access_unit_delimiter_NAL information shown in FIG.

  Here, when there is no error in access_unit_delimiter_NAL, the video decoding unit 40 can correctly determine a picture boundary based on the leak, so this process is terminated and the process proceeds to the process for the next TS packet. On the other hand, if access_unit_delimiter_NAL cannot be analyzed correctly and it is determined that there is an error, the contents of access_unit_delimiter_NAL are corrected to the contents shown in FIG. 4 (step 5k).

  The video decoding unit 40 decodes the video stream based on the video stream and boundary information given from the A / V separation unit 30, and obtains data for each picture. The video decoding unit 40 includes a NAL type decoding unit 41, an additional information decoding unit 42, a slice header decoding unit 43, a picture boundary determination unit 44, a picture boundary processing unit 45, and a slice data decoding unit 46.

  The NAL type decoding unit 41 determines whether the video stream supplied from the A / V separation unit 30 is a NAL indicating additional information or a NAL composed of slices including video data. The additional information decoding unit 42 and the slice data are transmitted to the slice header decoding unit 43.

  The additional information decoding unit 42 decodes additional information other than video data. For example, video screen size, bit rate, time information, or special information such as access_unit_delimiter_NAL.

The slice header decoding unit 43 decodes the header information (slice header) of each slice data in the video stream supplied from the A / V separation unit 30.
This slice header includes parameters necessary for determining a picture boundary such as frame_num. By giving these pieces of information to the picture boundary determining unit 44, it is determined whether or not this slice is information on a new picture. Is determined.

  When the boundary information is given from the boundary detection unit 30a, the picture boundary determination unit 44 determines that there is no error in the stream input to the video decoding unit 40 and that a picture boundary is included. Further, the picture boundary determination unit 44 determines that the picture boundary is a picture boundary even when boundary information is not given from the boundary detection unit 30a, even when the access_unit_delimiter_NAL is decoded by the additional information decoding unit 42. The picture boundary determination unit 44 does not trust the information of frame_num when information indicating that there is no error is obtained from the boundary detection unit 30a. These determination results are output to the picture boundary processing unit 45.

  The picture boundary processing unit 45 performs processing associated with the picture boundary when the picture boundary determination unit 44 determines that it is a picture boundary. For example, it is determined that the image data for the screen size obtained by the additional information decoding unit 42 has been decoded, and the content of the decoded picture buffer (DPB) for storing and displaying the decoded image is updated. If there is, display processing is performed. In addition, a parameter for each picture (for example, a macroblock number) is reset.

  The slice data decoding unit 46 decodes each slice data in the video stream input to the video decoding unit 40 in accordance with an instruction from the picture boundary processing unit 45 to obtain decoded data for each picture.

  As described above, in the decoding apparatus having the above configuration, the boundary detection unit 30a of the A / V separation unit 30 checks the PES header and the access_unit_delimiter_NAL in the video stream without error depending on whether or not an error is mixed in the TS packet. In order to check whether it has been inserted and correct it, the video decoding unit determines the picture boundary based on access_unit_delimiter_NAL.

  Therefore, according to the decoding apparatus having the above configuration, even when the video decoding unit 40 performs boundary determination based on access_unit_delimiter_NAL, it is possible to prevent access_unit_delimiter_NAL from being detected due to an error, and thus it is possible to correctly determine a picture boundary.

  In the second embodiment, when a picture boundary is detected in step 5h, the process for correcting access_unit_delimiter_NAL (steps 5j and 5k) is executed. In addition to this, for example, if the NAL control unit 30b does not detect a picture boundary in step 5h, it is confirmed whether or not an error has occurred by confirming that information indicating the picture boundary is not set in access_unit_delimiter_NAL. When an error has occurred, that is, when information indicating a picture boundary is set, a step of correcting access_unit_delimiter_NAL so that access_unit_delimiter_NAL does not indicate a picture boundary may be provided.

Further, the present invention is not limited to the above-described embodiment as it is, and can be embodied by modifying the constituent elements without departing from the gist thereof in the implementation stage. In addition, various inventions can be formed by appropriately combining a plurality of constituent elements disclosed in the embodiment. Further, for example, a configuration in which some components are deleted from all the components shown in the embodiment is also conceivable. Furthermore, you may combine suitably the component described in different embodiment.
In addition, it goes without saying that the present invention can be similarly implemented even if various modifications are made without departing from the gist of the present invention.

1 is a circuit block diagram showing a configuration of a first embodiment of a decoding apparatus according to the present invention. 6 is a flowchart for explaining a picture boundary detection operation in the A / V separation unit of the decoding apparatus shown in FIG. 1. The circuit block diagram which shows the structure of 2nd Embodiment of the decoding apparatus concerning this invention. The figure which shows the format of access_Unit_delimiter_NAL which shows a picture boundary explicitly. 4 is a flowchart for explaining a picture boundary detection operation in an A / V separation unit of the decoding apparatus shown in FIG. 3.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 10 ... A / V separation part, 10a ... Boundary detection part, 20 ... Video decoding part, 21 ... NAL type decoding part, 22 ... Additional information decoding part, 23 ... Slice header decoding part, 24 ... Picture boundary determination part, 25 ... Picture boundary processing unit, 26 ... slice data decoding unit, 30 ... A / V separation unit, 30a ... boundary detection unit, 30b ... NAL control unit, 40 ... video decoding unit, 41 ... NAL type decoding unit, 42 ... additional information decoding , 43 ... slice header decoding unit, 43 ... additional information decoding unit, 44 ... picture boundary determination unit, 45 ... picture boundary processing unit, 46 ... slice data decoding unit.

Claims (2)

Each picture data is divided into a plurality of slice data, a video stream in which the slice data is packetized, and a multiplexed stream in which an audio stream is multiplexed are input, and a decoding device for decoding the video stream,
The slice data is classified for each original picture data based on separation means for separating the video stream and the audio stream from the multiplexed stream, and information included in a header of a packet constituting the video stream. An AV separation unit comprising boundary information detection means for detecting boundary information for
A video decoding unit that classifies the slice data into the same picture data based on the boundary information notified from the boundary information detection unit of the AV separation unit, and decodes the slice data;
The video decoding unit further comprises identification information detecting means for detecting identification information for classifying the slice data for each same picture data from the header of the slice data, and based on the boundary information and the identification information, A decoding apparatus, wherein the slice data is classified into the same picture data and the slice data is decoded.   The video decoding unit decodes the slice data by classifying the slice data into the same picture data using the boundary information preferentially among the boundary information and the identification information. Item 4. The decoding device according to Item 1.

Images