JP2020061773A - Decoding device - Google Patents

Abstract

To enable elimination of a period in which no picture is displayed when a channel is switched by a user, thereby implementing seamless channel switching.SOLUTION: Presentation time information indicating presentation time of a first access unit in order of presentation and information for calculating the presentation time of the first access unit in order of encoding from the presentation time information indicating the presentation time of the first access unit in order of presentation in a group of pictures (GOP) which is an assembly of a plurality of access units (AUs) encoded by an inter-frame prediction encoding system are obtained from encoded data. A video signal included in the encoded data is decoded by using the presentation time information of the first access unit in order of presentation and the information for calculating the presentation time of the first access unit in order of encoding.SELECTED DRAWING: Figure 3

Description

  The present invention relates to a coding apparatus and a coding method for coding a video signal and an audio signal to generate a bitstream, and a decoding apparatus and a decoding method for decoding coded data multiplexed in the bitstream. is there.

  In Japanese digital broadcasting, as described in Non-Patent Document 1 below, a video stream and an audio stream which are encoded data of a video signal and an audio signal are MPEG-2 (Moving Picture Experts Group Phase-2). It is multiplexed and transmitted in the transport stream (TS) format, which is a system standard of. At this time, the encoding device multiplexes the encoded data of the metadata related to the video stream and the audio stream together with the video stream and the audio stream, and transmits the multiplexed data.

  In addition to the transport stream (TS) in MPEG-2, there is MMT (MPEG Media Transport) as a new transport method that is being standardized in MPEG, and MMT is one or more that compose one program. When transmitting a video component (video stream) and an audio component (audio stream), it is possible to perform transmission in different transmission modes (for example, broadcasting and communication) for each component.

Here, HEVC / H. H.265 (hereinafter referred to as "HEVC") is a new video coding method standardized by MPEG and ITU (International Telecommunications Union).
In HEVC, temporal hierarchical coding (scalable coding in the time direction) is introduced, and NAL (a unit including coding data necessary for decoding one picture) of an access unit (NAL) A hierarchy level can be designated for each Network Abstraction Layer) unit.

FIG. 8 is an explanatory diagram showing an example of temporal layer coding in HEVC.
In FIG. 8, TemporalID is identification information indicating the hierarchical level of each access unit (AU).
In the case of TemporalID = L ₀ , since there is only a layer of 0, temporal layer coding is not performed.
When TemporalID = L ₁ , the maximum layer is 1, and temporal layer coding is performed.
Similarly, when TemporalID = L ₂ and L ₃ , the maximum layers are layers ₂ and ₃ , respectively, and temporal layer coding is performed.

The content of the time hierarchical coding is publicly known, and thus a detailed description thereof will be omitted. However, as a restriction of the time hierarchical coding, an access unit (AU) having a hierarchical level higher than that of the access unit (AU) to be coded There is something that cannot be referred to.
In HEVC, temporal hierarchical coding is possible with a reference structure having a maximum hierarchical layer of 6.

FIG. 9 is an explanatory diagram showing an example of the picture structure.
In FIG. 9, IRAP is an IRAP (Intra random access point) picture specified by HEVC, and when decoding is started from the middle of a bitstream, pictures after the IRAP picture are normally displayed in the display order. Guaranteed to be decrypted.
GOP (Group Of Pictures) is capable of decoding all video signals of one or more access units when video signals of one or more access units (AU) are encoded by an interframe predictive coding method. It is a set of possible multiple access units (AU). That is, it is a set of an IRAP picture which is the first access unit (AU) in the encoding order and an access unit (AU) (pictures other than IRAP picture) following the IRAP picture.
Also, the SOP is a set of a picture having a hierarchical level 0, a picture at the head in the coding order, and pictures at a hierarchical level 1 or higher following the picture.
One sequence bit stream is composed of one or more GOPs, and one GOP is composed of one or more SOPs. In the example of FIG. 9, a GOP is composed of the SOP of L _{3 and} the SOP of L ₂ .

FIG. 10 is an explanatory diagram showing the coding order and display order of each picture coded with the picture structure of FIG.
In the head SOP, each picture following the IRAP picture, which is the head access unit (AU) in the coding order, is coded using the picture of the immediately preceding SOP as a reference image.
Therefore, in order to normally decode each picture following the IRAP picture, it is necessary to refer to the picture of the immediately preceding SOP, so that the picture of the immediately preceding SOP needs to be already decoded.
Therefore, when decoding is started from the IRAP picture of the first SOP, it is not possible to refer to the picture of the immediately preceding SOP (the picture of the immediately preceding SOP has not been decoded). Each subsequent picture cannot be decoded normally.
In HEVC, a picture that follows an IRAP picture in coding order and is earlier in display order than an IRAP picture is called an LP (Leading Picture), and if decoding starts from an IRAP picture, the LP will perform decoding normally. I can't.
Since the IRAP picture is coded by the intraframe coding method, the IRAP picture can be normally decoded even if the picture of the immediately previous SOP is not decoded.

For example, in digital broadcasting, a plurality of bit streams are simultaneously delivered, and the bit stream to be displayed is switched by the user switching the channel.
Since IRAP pictures are periodically inserted in a plurality of bitstreams, the display video can be switched by starting decoding from one of the IRAP pictures when the user switches channels.

STD-B32 (Standard for digital broadcasting established by ARIB (The Association of Radio Industries and Businesses))

Since the conventional coding apparatus is configured as described above, when performing temporal hierarchical coding, the larger the maximum hierarchical level, the greater the number of pictures that make up the SOP. Therefore, when the decoding device starts decoding from an IRAP picture, the number of LPs that cannot be decoded normally increases. Therefore, when the bit stream to be displayed is switched by the channel switching by the user, There is a problem that it takes a long time to display a picture of a bitstream, and the time during which no picture is displayed becomes long.
That is, when the bit stream to be displayed is switched by the channel switching by the user, what is displayed from the display time of the first LP (picture B25 in the example of FIG. 10) to the display time of the IRAP picture. Since no picture is displayed, there is a problem in that the larger the number of pictures forming the SOP, the longer the time during which no picture is displayed.

  The present invention has been made to solve the above problems, and it is possible to realize seamless channel switching by eliminating the time when no picture is displayed when the channel is switched by the user. An object is to obtain a device, a decoding device, an encoding method and a decoding method.

  A decoding device according to the present invention is a decoding device that decodes coded data of a video signal transmitted by a transmission system defined by MMT (MPEG Media Transport), and uses the inter-frame predictive coding system from the coded data. Encoding from presentation time information indicating the presentation time of the first access unit in the presentation order and presentation time information indicating the presentation time of the first access unit in the presentation order in a GOP that is a set of encoded multiple access units Control information decoding means for obtaining information for calculating the presentation time of the leading access unit in order, presentation time information of the leading access unit in the presentation order acquired by the control information decoding means, and in coding order The video signal included in the encoded data is decoded by using the information for calculating the presentation time of the head access unit And decryption means.

  According to the present invention, there is provided a decoding device for decoding encoded data of a video signal transmitted by a transmission method defined by MMT (MPEG Media Transport), wherein the encoded data is encoded by an interframe predictive encoding method. In the GOP, which is a set of a plurality of access units that have been presented, the presentation time information indicating the presentation time of the first access unit in the presentation order and the presentation time information indicating the presentation time of the first access unit in the presentation order are encoded in the encoding order Control information decoding means for obtaining information for calculating the presentation time of the head access unit, presentation time information of the head access unit in the presentation order acquired by the control information decoding means, and head information in the encoding order. Decoding means for decoding the video signal included in the encoded data by using the information for calculating the presentation time of the access unit With the above, there is an effect that it is possible to realize seamless channel switching by eliminating the time during which no picture is displayed when the channel is switched by the user.

It is a block diagram which shows the encoding device by Embodiment 1 of this invention. 3 is a flowchart showing the processing contents (encoding method) of the encoding device according to the first embodiment of the present invention. It is a block diagram which shows the decoding apparatus by Embodiment 1 of this invention. 3 is a flowchart showing the processing contents (decoding method) of the decoding device according to the first embodiment of the present invention. It is explanatory drawing which shows the outline | summary of the encoded data at the time of transmitting a bit stream by MMT. It is explanatory drawing which shows the structural example of MPU. It is explanatory drawing which shows a HEVC picture structure descriptor. It is an explanatory view showing an example of time hierarchy coding in HEVC. It is explanatory drawing which shows an example of a picture structure. FIG. 10 is an explanatory diagram showing an encoding order and a presentation order of each picture encoded with the picture structure of FIG. 9.

Embodiment 1.
1 is a block diagram showing an encoding apparatus according to Embodiment 1 of the present invention.
In FIG. 1, when a voice encoding unit 1 is provided with a digital voice signal, the voice encoding unit 1 encodes the voice signal in units of voice access units (AU) by a method such as MPEG-4 audio and outputs the voice signal. The audio stream that is the encoded data of is generated, and the processing of encoding the metadata regarding the audio stream is performed.
The audio MMTP payload generation unit 2 executes a process of generating an audio MMTP payload composed of the metadata encoded by the audio encoding unit 1 and the encoded data of the audio signal in access unit (AU) units.

When the HEVC encoding unit 3 is supplied with a digital video signal, the HEVC encoding unit 3 encodes the video signal in units of video access units (AU) by the HEVC method, and generates a video stream which is encoded data of the video signal. At the same time, a process of encoding metadata regarding the video stream is performed.
The video MMTP payload generation unit 4 performs a process of generating a video MMTP payload composed of the metadata coded by the HEVC coding unit 3 and the coded data of the video signal in access unit (AU) units. The HEVC encoder 3 and the video MMTP payload generator 4 constitute a video encoder.
Here, as the metadata related to the audio stream and the video stream, for example, time information indicating the DTS (decoding time) or PTS (presentation time) of each access unit (AU) can be described.

The control information encoding unit 5 encodes control information called a PA message defined by MMT as control information regarding the audio stream generated by the audio encoding unit 1 and the video stream generated by the HEVC encoding unit 3. Perform the processing to do.
The PA message describes information about one or more video components (video streams) and audio components (audio streams) that form one program (called a package in MMT). In MMT, video components and audio components are called assets.
Specifically, an asset ID for identifying the asset, an asset type for identifying the type of the asset (type such as HEVC format video stream and MPEG-4 audio format audio stream), and MPU (Media Processing Unit) of each asset. Among the constituent access units (AU), MPU time stamp descriptor (presentation time information) indicating the presentation time (display time) of the first access unit (AU) in presentation order (display order), HEVC picture structure Descriptors, packet IDs indicating MMTP packets storing encoded data and metadata of each asset, and the like are included in the PA message by the number of assets that make up the package.
The MPU is composed of one or more access units (AUs), and is a unit that can perform video and audio decoding processing by the MPU alone. When the video signal of one or more access units (AU) is encoded by the interframe predictive coding method, the MPU should decode all the video signals of the one or more access units (AU). It becomes the same unit as a GOP which is a set of a plurality of access units (AU) capable of performing.

  In the HEVC picture structure descriptor, the number of access units (AU) that are earlier in presentation order than the first access unit (AU) in the encoding order among the access units (AU) that make up the MPU (the number of LPs). (Num_of_leading_picture) indicating the number), picture type information (rap_type) indicating the coding method of the NAL unit (null unit) that constitutes the first access unit (AU) in the coding order, and LP. The picture type information (nal_unit_type_of_leading_picture) indicating the encoding method of the NAL unit is described.

The control MMTP payload generator 6 carries out a process of generating a control MMTP payload composed of encoded data of the control information encoded by the control information encoder 5.
The control information encoding unit 5 and the control MMTP payload generation unit 6 constitute control information encoding means.

  The MMTP packet generator 7 includes the audio MMTP payload generated by the audio MMTP payload generator 2, the video MMTP payload generated by the video MMTP payload generator 4, and the control MMTP payload generated by the control MMTP payload generator 6. Is multiplexed to generate an MMTP packet forming a bitstream. When this MMTP packet is generated, a predetermined MMTP header is added, and this MMTP header includes a packet ID assigned according to the type of encoded data included in the MMTP payload. The MMTP packet generator 7 constitutes a multiplexing means.

In the example of FIG. 1, a voice coding unit 1, a voice MMTP payload generation unit 2, a HEVC coding unit 3, a video MMTP payload generation unit 4, a control information coding unit 5, and a control MMTP payload, which are components of the coding device. It is assumed that each of the generation unit 6 and the MMTP packet generation unit 7 is configured by dedicated hardware (for example, a semiconductor integrated circuit mounting a CPU, a one-chip microcomputer, etc.). The computerization device may be configured by a computer.
When the encoding device is configured by a computer, a voice encoding unit 1, a voice MMTP payload generating unit 2, a HEVC encoding unit 3, a video MMTP payload generating unit 4, a control information encoding unit 5, a control MMTP payload generating unit 6, and A program describing the processing contents of the MMTP packet generation unit 7 may be stored in the memory of the computer, and the CPU of the computer may execute the program stored in the memory.
FIG. 2 is a flowchart showing the processing contents (encoding method) of the encoding device according to the first embodiment of the present invention.

FIG. 3 is a block diagram showing a decoding device according to the first embodiment of the present invention.
In FIG. 3, the stream selection unit 11 includes a bitstream (a bitstream including MMTP packets) output from a plurality of encoding devices (encoding device of FIG. 1 or an encoding device corresponding to the encoding device of FIG. 1). The process of selecting a bitstream to be presented from among the above) and outputting the bitstream to the MMTP packet analysis unit 12 is performed.
Further, when the instruction to switch the bitstream to be presented is given, the stream selection unit 11 selects the bitstream after switching from the bitstreams output from the plurality of encoding devices, and selects the bitstream. The process of outputting to the MMTP packet analysis unit 12 and also outputting the bit stream before switching to the MMTP packet analysis unit 12 until the presentation time calculated by the control MMTP payload processing unit 13 is reached. The stream selection unit 11 constitutes a bitstream selection means.

  The MMTP packet analysis unit 12 analyzes the MMTP header of the MMTP packet forming the bit stream output from the stream selection unit 11, acquires the packet ID included in the MMTP header, and the packet ID is MMTP. If it indicates that the encoded data included in the payload is control information (PA message, HEVC picture structure descriptor), the control MMTP payload, which is the MMTP payload included in the MMTP packet, is changed to the control MMTP payload. Output to the processing unit 13. On the other hand, if the packet ID indicates that the encoded data included in the MMTP payload is an audio signal or a video signal, a process of outputting the MMTP packet to the asset separation unit 14 is performed.

The control MMTP payload processing unit 13 performs a decoding process of the encoded data included in the control MMTP payload output from the MMTP packet analysis unit 12, and a PA message that is control information and an HEVC included in the PA message. Decode the picture structure descriptor.
Further, the control MMTP payload processing unit 13 presents the presentation time of the first access unit (AU) in the presentation order indicated by the MPU time stamp descriptor described in the PA message and the number information described in the HEVC picture structure descriptor. (Num_of_leading_picture) The presentation time of the first access unit (AU) in the encoding order from the number of access units (AU) in the presentation order earlier than the first access unit (AU) in the encoding order. The process of calculating is carried out. The first access unit (AU) in the encoding order is the first SOP IRAP picture. The control MMTP payload processing unit 13 constitutes a presentation time calculation means.

  The asset separation unit 14 refers to the asset ID, the asset type, and the packet ID described in the PA message decrypted by the control MMTP payload processing unit 13 and includes them in the MMTP packet output from the MMTP packet analysis unit 12. It is specified whether the MMTP payload that is present is an audio MMTP payload or a video MMTP payload. If it is an audio MMTP payload, the audio MMTP payload included in the MMTP packet is extracted and the audio MMTP payload is converted to audio. If the video MMTP payload is output to the MMTP payload processing unit 15, the video MMTP payload included in the MMTP packet is extracted, and the video MMTP payload is output to the video MMTP payload processing unit 19. Hodokosuru.

The audio MMTP payload processing unit 15 reconfigures the MFU (Media Fragment Unit) or MPU of the audio stream from the audio MMTP payload output from the asset separation unit 14 so that the audio stream decoding unit 17 in the subsequent stage can perform decoding. A process of generating an audio elementary stream (audio ES) and storing the audio ES in the audio ES buffer 16 is performed. The MFU is a unit smaller than the MPU, and one access unit (AU) or one NAL unit can be defined as one MFU.
In addition, the audio MMTP payload processing unit 15 extracts metadata regarding the audio stream included in the audio MMTP payload output from the asset separation unit 14 and stores the metadata in the audio ES buffer 16.
The audio ES buffer 16 is a memory that temporarily stores the audio ES and metadata.

The audio stream decoding unit 17 extracts the metadata from the audio ES buffer 16 and the time information described in the metadata (information indicating the DTS (decoding time) or PTS (presentation time) of each access unit (AU)). Is executed.
When the DTS (decoding time) of each access unit (AU) is reached, the audio stream decoding unit 17 takes out the audio ES from the audio ES buffer 16, decodes the audio signal of the access unit (AU), and decodes the audio signal. A process of storing the audio signal and PTS (presentation time) in the audio data buffer 18 is performed.
The audio data buffer 18 is a memory that temporarily stores the audio signal decoded by the audio stream decoding unit 17 and the PTS (presentation time).

The video MMTP payload processing unit 19 reconfigures the MFU or MPU of the video stream from the video MMTP payload output from the asset separation unit 14 so that the HEVC elementary stream (HEVC) in a format that can be decoded by the HEVCES decoding unit 21 in the subsequent stage. ES) is generated and the HEVC elementary stream is stored in the HEVC ES buffer 20.
In addition, the video MMTP payload processing unit 19 extracts metadata about the video stream included in the video MMTP payload output from the asset separation unit 14 and stores the metadata in the HEVCES buffer 20.
The HEVCES buffer 20 is a memory that temporarily stores the HEVC elementary stream and metadata.

The HEVCES decoding unit 21 extracts the metadata from the HEVCES buffer 20 and decodes the time information (the information indicating the DTS (decoding time) and PTS (presenting time) of each access unit (AU)) described in the metadata. Perform the processing to do.
When the HEVCES decoding unit 21 reaches the DTS (decoding time) of each access unit (AU), it extracts the HEVC elementary stream from the HEVCES buffer 20, decodes the video signal of the access unit (AU), and decodes it. A process of storing the decoded image, which is a video signal, and the PTS (presentation time) in the decoded image buffer 22 is performed.
The decoded image buffer 22 is a memory that temporarily stores the decoded image of each access unit (AU) decoded by the HEVCES decoding unit 21 and the PTS (presentation time).
The video MMTP payload processing unit 19, HEVCES buffer 20, HEVCES decoding unit 21, and decoded image buffer 22 constitute a video decoding means.

In the example of FIG. 3, the stream selection unit 11, the MMTP packet analysis unit 12, the control MMTP payload processing unit 13, the asset separation unit 14, the audio MMTP payload processing unit 15, the audio ES buffer 16, and the audio stream which are the components of the decoding device. Each of the decoding unit 17, the audio data buffer 18, the video MMTP payload processing unit 19, the HEVCES buffer 20, the HEVCES decoding unit 21, and the decoded image buffer 22 is dedicated hardware (other than the buffer, for example, a semiconductor mounting a CPU). An integrated circuit, a one-chip microcomputer, or the like is assumed, but the decoding device may be a computer.
When the decoding device is configured by a computer, the audio ES buffer 16, the audio data buffer 18, the HEVCES buffer 20, and the decoded image buffer 22 are configured on the internal memory or the external memory of the computer, and the stream selection unit 11 and the MMTP packet analysis unit. 12, a control MMTP payload processing unit 13, an asset separation unit 14, an audio MMTP payload processing unit 15, an audio stream decoding unit 17, a video MMTP payload processing unit 19 and a HEVCES decoding unit 21 and a program describing the processing contents of the computer. It may be stored in the memory and the CPU of the computer may execute the program stored in the memory.
FIG. 4 is a flowchart showing the processing contents (decoding method) of the decoding device according to the first embodiment of the present invention.

Next, the operation will be described.
The processing contents of the first encoding device will be described.
When a digital audio signal is given, the audio encoding unit 1 encodes the audio signal for each audio access unit (AU) by a method such as MPEG-4 audio, and encodes the audio signal. An audio stream that is data is generated, and metadata regarding the audio stream is encoded (step ST1 in FIG. 2).
When a digital video signal is given, the HEVC encoding unit 3 encodes the video signal for each video access unit (AU) by the HEVC system, and generates a video stream that is encoded data of the video signal. At the same time, the metadata regarding the video stream is encoded (step ST2).

Here, FIG. 5 is an explanatory diagram showing an outline of encoded data when a bit stream is transmitted by MMT.
In FIG. 5, an access unit (AU) is a unit including encoded data necessary for decoding one picture in the case of video, and is composed of one or more samples as an encoding unit in the case of audio. It is a frame to be played.
The NAL unit is a HEVC coding unit, and one access unit (AU) is composed of one or more NAL units.
The MPU is composed of one or more access units and serves as a unit capable of performing video and audio decoding processing by the MPU alone. When the video signal of one or more access units (AU) is encoded by the interframe predictive coding method, the MPU should decode all the video signals of the one or more access units (AU). It becomes the same unit as a GOP which is a set of a plurality of access units (AU) capable of performing.
The MFU is a unit smaller than the MPU, and one access unit (AU) or one NAL unit can be defined as one MFU.

FIG. 6 is an explanatory diagram showing a configuration example of the MPU.
In FIG. 6, the MPU metadata describes metadata related to the MPU, such as time information indicating DTS (decoding time) and PTS (presentation time) of each access unit (AU) included in the MPU. Can be described.
The movie fragment metadata (MF meta) describes metadata associated with encoded data (sample data) of one access unit (AU). For example, when the encoded data of the access unit (AU) is stored in a file format, the address at which the encoded data is stored, the data length of the encoded data, and the access unit (AU) are stored for each access unit (AU). ) Is included in the information about the presentation time.
The MPU metadata, movie fragment metadata, MFU, and MMT control information are transmitted as MMTP packets. The MMTP packet is composed of an MMTP header and an MMTP payload.

When the audio MMTP payload generator 2 receives the encoded data of metadata (MPU metadata, MF meta, etc.) and the encoded data of the audio signal of each access unit (AU) from the audio encoder 1, the audio MMTP payload generator 2 receives the MPU. An audio MMTP payload including encoded data of unit MPU metadata, encoded data of access unit (AU) MF meta, and encoded data of audio signal (sample data) is generated (step ST3).
When the video MMTP payload generation unit 4 receives the encoded data of metadata (MPU metadata, MF meta, etc.) and the encoded data of the video signal in access unit (AU) units from the HEVC encoding unit 3, the MPU receives the MPU. A video MMTP payload composed of encoded data of MPU metadata of a unit, encoded data of MF meta of an access unit (AU) and encoded data (sample data) of a video signal is generated (step ST4).

The control information encoding unit 5 encodes the control information regarding the audio stream generated by the audio encoding unit 1 and the video stream generated by the HEVC encoding unit 3 (step ST5).
As the control information regarding the audio stream and the video stream, for example, a PA message or HEVC picture structure descriptor defined by MMT is encoded.
As described above, the PA message describes information about one or more video components (video streams) and audio components (audio streams) that form one program (called a package in MMT).
That is, in the PA message, an asset ID that identifies an asset (video stream, audio stream) generated by the audio encoding unit 1 and the HEVC encoding unit 3, an asset type that identifies the type of asset, and an MPU of each asset. Among the constituent access units (AU), the MPU time stamp descriptor indicating the presentation time of the first access unit (AU) in the presentation order, and the MMTP packet storing the encoded data and metadata of each asset. Is described.

FIG. 7 is an explanatory diagram showing the HEVC picture structure descriptor.
In the HEVC picture structure descriptor, as shown in FIG. 7, among the access units (AU) forming the MPU, the access unit (AU) whose presentation order is earlier than the first access unit (AU) in the encoding order. The number information (num_of_leading_picture) indicating the number (number of LPs) is described.
Also, picture type information (rap_type) indicating the encoding method of the NAL unit that constitutes the first access unit (AU) in encoding order, and the picture type that indicates the encoding method of the NAL unit that constitutes LP. Information (nal_unit_type_of_leading_picture) and the like are described.

Upon receiving the encoded data of the control information from the control information encoding unit 5, the control MMTP payload generation unit 6 generates the control MMTP payload including the encoded data of the control information (step ST6).
The MMTP packet generator 7 includes an audio MMTP payload generated by the audio MMTP payload generator 2, a video MMTP payload generated by the video MMTP payload generator 4, and a control MMTP payload generated by the control MMTP payload generator 6. And are multiplexed to generate an MMTP packet forming a bitstream (step ST7).
When this MMTP packet is generated, a predetermined MMTP header is added, and this MMTP header includes a packet ID assigned according to the type of encoded data included in the MMTP payload.

Next, the processing contents of the decoding device will be described.
The stream selection unit 11 is given a bitstream (bitstream consisting of MMTP packets) output from a plurality of encoding devices (encoding device of FIG. 1 or encoding device corresponding to the encoding device of FIG. 1). To be
The stream selection unit 11 selects a bitstream of a channel designated by the user (bitstream to be presented) from the plurality of bitstreams and outputs the bitstream to the MMTP packet analysis unit 12.

Upon receiving the bitstream from the stream selection unit 11, the MMTP packet analysis unit 12 analyzes the MMTP header of the MMTP packet forming the bitstream and acquires the packet ID included in the MMTP header.
If the packet ID indicates that the encoded data included in the MMTP payload is control information (PA message), the MMTP packet analysis unit 12 controls the MMTP payload included in the MMTP packet. The MMTP payload is output to the control MMTP payload processing unit 13.
On the other hand, if the packet ID indicates that the encoded data included in the MMTP payload is an audio signal or a video signal, the MMTP packet is output to the asset separation unit 14.

  When the control MMTP payload processing unit 13 receives the control MMTP payload from the MMTP packet analysis unit 12, the control MMTP payload processing unit 13 performs a decoding process on the encoded data included in the control MMTP payload, and a PA message and a PA message as control information Decode the HEVC picture structure descriptor contained in

When the control MMTP payload processing unit 13 decodes the PA message, the asset separation unit 14 refers to the asset ID, the asset type, and the packet ID described in the PA message, and outputs the MMTP output from the MMTP packet analysis unit 12. It specifies whether the MMTP payload included in the packet is an audio MMTP payload or a video MMTP payload.
If the MMTP payload included in the MMTP packet output from the MMTP packet analysis unit 12 is a voice MMTP payload, the asset separating unit 14 extracts the voice MMTP payload included in the MMTP packet, and extracts the voice. The MMTP payload is output to the voice MMTP payload processing unit 15.
If the MMTP payload included in the MMTP packet output from the MMTP packet analysis unit 12 is a video MMTP payload, the asset separation unit 14 extracts the video MMTP payload included in the MMTP packet and extracts the video. The MMTP payload is output to the video MMTP payload processing unit 19.

Upon receiving the audio MMTP payload from the asset separation unit 14, the audio MMTP payload processing unit 15 reconfigures the MFU or MPU of the audio stream from the audio MMTP payload so that the audio stream decoding unit 17 in the subsequent stage can perform decoding. The audio elementary stream (audio ES) is generated, and the audio ES is stored in the audio ES buffer 16.
Since the process itself for generating the audio ES from the audio MMTP payload is a known technique, its detailed description is omitted.
Further, the audio MMTP payload processing unit 15 extracts metadata regarding the audio stream included in the audio MMTP payload output from the asset separation unit 14, and stores the metadata in the audio ES buffer 16.

The audio stream decoding unit 17 extracts the metadata from the audio ES buffer 16 and indicates the time information (DTS (decoding time) or PTS (presentation time) of each access unit (AU) described in the metadata. ) Is decrypted.
The audio stream decoding unit 17 refers to the decoded DTS to grasp the decoding time of each access unit (AU), and at the decoding time of each access unit (AU), extracts the audio ES from the audio ES buffer 16. The audio signal of the access unit (AU) is decoded, and the decoded audio signal and PTS (presentation time) are stored in the audio data buffer 18.
As a result, an external reproduction device (not shown) can reproduce the audio signal at the presentation time by extracting the audio signal and PTS (presentation time) stored in the audio data buffer 18.

Upon receiving the video MMTP payload from the asset separation unit 14, the video MMTP payload processing unit 19 reconfigures the MFU or MPU of the video stream from the video MMTP payload, so that the HEVCES decoding unit 21 in the subsequent stage can perform decoding. The HEVC elementary stream (HEVC ES) is generated, and the HEVC elementary stream is stored in the HEVC ES buffer 20.
Since the process itself for generating the HEVC elementary stream from the video MMTP payload is a known technique, its detailed description is omitted.
In addition, the video MMTP payload processing unit 19 extracts metadata regarding the video stream included in the video MMTP payload output from the asset separation unit 14, and stores the metadata in the HEVCES buffer 20.

The HEVCES decoding unit 21 extracts the metadata from the HEVCES buffer 20 and outputs the time information (the information indicating the DTS (decoding time) and PTS (presentation time) of each access unit (AU)) described in the metadata. Decrypt.
The HEVCES decoding unit 21 grasps the decoding time of each access unit (AU) with reference to the decoded DTS, and when the decoding time of each access unit (AU) comes, extracts the HEVC elementary stream from the HEVCES buffer 20. The video signal of the access unit (AU) is decoded, and the decoded image and PTS (presentation time) which are the decoded video signal are stored in the decoded image buffer 22.
Thus, an external reproduction device (not shown) can reproduce the decoded image at the presentation time by extracting the decoded image and PTS (presentation time) stored in the decoded image buffer 22.

When an external playback device (not shown) is playing back a decoded image and an audio signal, when the user performs a channel switching operation using a remote controller or the like, a command to switch the video stream to be presented (this switching command Includes information indicating the channel after switching) to the stream selection unit 11.
When receiving a channel switching command from the outside (step ST11: Yes in FIG. 4), the stream selection unit 11 selects the bitstream of the channel after switching indicated by the switching command from the plurality of bitstreams. And outputs the bit stream to the MMTP packet analysis unit 12 (step ST12).
Further, the stream selection unit 11 outputs the bit stream from the control MMTP payload processing unit 13 in order to realize seamless channel switching by eliminating the time when no decoded image is displayed when the channel is switched by the user. Until the stop command is received (step ST13: No), the bit stream of the channel before switching is also continuously output to the MMTP packet analysis unit 12 (step ST14). As will be described later, the bitstream output stop command is output when the current time reaches the presentation time of the head access unit (AU) in the encoding order.
When receiving the bitstream output stop command from the control MMTP payload processing unit 13, the stream selection unit 11 stops the output of the bitstream of the channel before switching and only the bitstream of the channel after switching is processed by the MMTP packet analysis unit. Output to 12.

When the MMTP packet analysis unit 12 receives the bit stream of the channel after switching from the stream selection unit 11, the MMTP packet analysis unit 12 receives the control MMTP payload included in the MMTP packet forming the bit stream as before switching the channel. The control MMTP payload processing unit 13 outputs the audio MMTP payload or the video MMTP payload included in the MMTP packet forming the bitstream to the asset separation unit 14.
Further, when the MMTP packet analysis unit 12 receives the bitstream of the channel before switching from the stream selection unit 11, the MMTP packet analysis unit 12 configures the bitstream of the channel before switching by a process parallel to the bitstream of the channel after switching. The control MMTP payload included in the MMTP packet is output to the control MMTP payload processing unit 13, and the audio MMTP payload or the video MMTP payload included in the MMTP packet forming the bitstream is transferred to the asset separation unit 14. Output to.

Upon receiving the control MMTP payload related to the bit stream of the channel after switching from the MMTP packet analysis unit 12, the control MMTP payload processing unit 13 receives the encoded data included in the control MMTP payload as before the channel switching. Is executed to decode the PA message which is the control information and the HEVC picture structure descriptor included in the PA message (step ST15).
When the control MMTP payload processing unit 13 decodes the PA message and the HEVC picture structure descriptor included in the PA message, the head access unit (AU) in the presentation order indicated by the MPU time stamp descriptor described in the PA message. Presentation time and the number of access units (AU) that are earlier in presentation order than the first access unit (AU) in the encoding order indicated by the number information (num_of_leading_picture) described in the HEVC picture structure descriptor (number of LPs) From this, the presentation time of the first access unit (AU) in the encoding order is calculated (step ST16).

In the example of FIG. 10, the first access unit (AU) in the coding order is IRAP32, and the first access unit (AU) in the presentation order is B25. Further, the number of access units (AUs) (the number of LPs) whose presentation order is earlier than that of the IRAP 32 is seven.
Therefore, if the presentation time of B25, which is the first access unit (AU) in the presentation order, is, for example, 18:00:00 and the frame rate is 120 images / 1 second, the presentation time of IRAP32 is B25. This is 58 msec (= 7/120) after the presentation time (18:00:00).
When the current time reaches the presentation time of the first access unit (AU) in the encoding order (in the case of step ST17: Yes), the control MMTP payload processing unit 13 selects a stream stop command for outputting the bit stream of the channel before switching. It is output to the section 11 (step ST18).

When the asset separation unit 14 receives the MMTP packet related to the channel after switching from the MMTP packet analysis unit 12, the voice MMTP payload processing unit 15 receives the voice MMTP payload included in the MMTP packet as in the case before the channel switching. The video MMTP payload included in the MMTP packet is output to the video MMTP payload processing unit 19.
When the asset separation unit 14 receives the MMTP packet related to the channel before switching from the MMTP packet analysis unit 12, the asset separation unit 14 converts the MMTP packet related to the channel before switching into the MMTP packet related to the channel before switching by parallel processing. The included audio MMTP payload is output to the audio MMTP payload processing unit 15, and the video MMTP payload included in the MMTP packet is output to the video MMTP payload processing unit 19.

When the video MMTP payload processing unit 19 receives the video MMTP payload relating to the channel before switching from the asset separation unit 14 (before the current time reaches the presentation time of the first access unit (AU) in the presentation order), the channel switching is performed. As before, a HEVC elementary stream is generated from the video MMTP payload, the HEVC elementary stream is stored in the HEVCES buffer 20, and metadata regarding the video stream included in the video MMTP payload is extracted. , And stores the metadata in the HEVCES buffer 20 (step ST19).
Further, when the video MMTP payload processing unit 19 receives the video MMTP payload related to the channel after switching from the asset separation unit 14, the video MMTP payload processing is performed in parallel with the processing on the bit stream of the channel before switching, and the video related to the channel after switching. The HEVC elementary stream is generated from the MMTP payload, the HEVC elementary stream is stored in the HEVCES buffer 20, the metadata regarding the video stream included in the video MMTP payload is extracted, and the metadata is extracted into the HEVCES buffer. 20 (step ST20).

The HEVCES decoding unit 21 takes out the metadata from the HEVCES buffer 20 as in the case before the channel switching, and the time information (DTS (decoding time) or PTS (decoding time) of each access unit (AU) described in the metadata. Information indicating the presentation time) is decrypted.
As a result, the HEVCES decoding unit 21 recognizes the decoding time of each access unit (AU) by referring to the decoded DTS, but the access unit that can be first decoded by IRAP for the channel after switching. (AU) (in the GOP configuration of FIG. 10, the access unit (AU) of IRAP 32), it is impossible to decode the video signal of the access unit (AU) of LP whose presentation order is earlier than that of IRAP. Therefore, for the channel after switching, the video signal of any access unit (AU) cannot be decoded and displayed until the presentation time of IRAP.

Therefore, the HEVCES decoding unit 21 decodes the video signal of the access unit (AU) related to the channel before switching until the presentation time of the IRAP related to the channel after switching, and the decoded video signal. The decoded image and PTS (presentation time) are stored in the decoded image buffer 22.
This allows an external playback device (not shown) to play back the decoded image for the channel before switching until the presentation time of the IRAP for the channel after switching comes.
Therefore, it is possible to realize the seamless channel switching by eliminating the time when no decoded image is displayed when the channel is switched by the user.

When the audio MMTP payload processing unit 15 receives the audio MMTP payload related to the channel before switching from the asset separation unit 14 (before the current time reaches the presentation time of the first access unit (AU) in the presentation order), the channel switching is performed. As before, the audio ES is generated from the audio MMTP payload, the audio ES is stored in the audio ES buffer 16, the metadata regarding the audio stream included in the audio MMTP payload is extracted, and the metadata thereof is extracted. The data is stored in the audio ES buffer 16.
When the audio MMTP payload processing unit 15 receives the audio MMTP payload related to the channel after switching from the asset separation unit 14, the audio MMTP payload processing unit 15 performs the processing related to the bit stream of the channel before switching and the audio related to the channel after switching. The voice ES is generated from the MMTP payload, the voice ES is stored in the voice ES buffer 16, the metadata regarding the voice stream included in the voice MMTP payload is extracted, and the metadata is stored in the voice ES buffer 16. Store.

The audio stream decoding unit 17 extracts the metadata from the audio ES buffer 16 in the same manner as before switching the channel, and outputs the time information (DTS (decoding time) of each access unit (AU)) described in the metadata. Information indicating PTS (presentation time) is decoded.
The audio stream decoding unit 17 refers to the decoded DTS to grasp the decoding time of each access unit (AU), and at the decoding time of each access unit (AU), extracts the audio ES from the audio ES buffer 16. The audio signal of the access unit (AU) is decoded, and the decoded audio signal and PTS (presentation time) are stored in the audio data buffer 18.
As a result, an external reproduction device (not shown) can reproduce the audio signal at the presentation time by extracting the audio signal and PTS (presentation time) stored in the audio data buffer 18.

  Here, similarly to the HEVCES decoding unit 21, the audio stream decoding unit 17 outputs the audio signal of the access unit (AU) related to the channel before switching until the presentation time of the IRAP related to the channel after switching. Although it is assumed that the audio signal is decoded and the decoded audio signal and PTS (presentation time) are stored in the audio data buffer 18, the audio signal is encoded by the interframe predictive encoding method like the video signal. Since it is not performed, the audio signal of the LP access unit (AU), which is presented earlier than IRAP, can be decoded. Therefore, even before the presentation time of the IRAP related to the channel after switching, the access related to the channel after switching is performed without decoding the audio signal of the access unit (AU) related to the channel before switching. The audio signal of the unit (AU) may be decoded.

  The decoding device repeatedly executes the processes of steps ST11 to ST20 until the decoding process is completed (step ST21).

  As is clear from the above, according to the first embodiment, when the video signals of one or more access units (AU) are coded by the interframe predictive coding method, one or more access units (AU) Presentation time information indicating the presentation time of the first access unit (AU) in the presentation order and the encoding order for each GOP that is a set of a plurality of access units (AU) capable of decoding all the video signals of Since the control information including the number information indicating the number of access units (AU) that are presented earlier than the first access unit (AU) is encoded, the channel is switched by the user on the decoding side. At the same time, there is an effect that a coding device capable of realizing seamless channel switching can be obtained by eliminating the time when no decoded image is displayed. .

  Further, according to the first embodiment, when the instruction to switch the bit stream to be presented is given, the stream selection unit 11 selects the bit after switching from the bit streams output from the plurality of encoding devices. A stream is selected, the bit stream is output to the MMTP packet analysis unit 12, and the switching time before switching is reached until the presentation time calculated by the control MMTP payload processing unit 13 (until a bit stream output stop command is received). The bit stream is also continuously output to the MMTP packet analysis unit 12, and the HEVCES decoding unit 21 decodes the video signal for each access unit from the coded data of the video signal multiplexed in the bit stream after the switching, and the control MMTP. It becomes the presentation time calculated by the payload processing unit 13. During this period, the video signal for each access unit is decoded from the coded data of the video signal multiplexed in the bit stream before switching, so what kind of decoding is performed when the channel is switched by the user? There is an effect that a decoding device capable of realizing seamless channel switching can be obtained by eliminating the time when no image is displayed.

Embodiment 2.
In the above-described first embodiment, the HEVCES decoding unit 21 decodes the video signal of the access unit (AU) related to the channel before switching until the presentation time of the IRAP related to the channel after switching, and decodes the decoded video signal. Although the decoded image and the PTS (presentation time) which are the video signals stored are stored in the decoded image buffer 22, the access unit (AU) of the LP access unit (AU) related to the channel after switching is also shown. Depending on the encoding method of the NAL unit that composes (1), decoding may be possible even before IRAP decoding.
Whether or not the access unit (AU) of the LP can be decoded even before IRAP decoding is configured by referring to the picture type information (nal_unit_type_of_leading_picture) described in the HEVC picture structure descriptor. This can be determined because the encoding method of the NAL unit in use is known. For example, if the coding method of the NAL unit forming the LP is an intraframe coding method, it is possible to decode even if the video signal of the access unit (AU) of the immediately previous SOP is not decoded. It is possible.

If the HEVCES decoding unit 21 can decode the LP access unit (AU) related to the switched channel even before IRAP decoding, the HEVCES decoding unit 21 decodes the video signal of the access unit (AU). Then, the decoded image which is the decoded video signal and the PTS (presentation time) are stored in the decoded image buffer 22.
For example, in the GOP structure of FIG. 10, LPs B25, B25, B27, B28 cannot be decoded, but if B29, B30, B31 can be decoded, the video signals of B29, B30, B31 are decoded, and The decoded image, which is the decoded video signal, and the PTS (presentation time) are stored in the decoded image buffer 22.
As a result, the external playback apparatus (not shown) decodes the channel before switching at the presentation time of B25, B25, B27, B28 until the presentation time of the IRAP related to the channel after switching. The image is reproduced, and at the presentation time of B29, B30, B31, the decoded image of the channel after switching can be reproduced.

  It should be noted that, within the scope of the invention, the invention of the present application is capable of freely combining the respective embodiments, modifying any constituent element of each embodiment, or omitting any constituent element in each embodiment. .

  1 audio coding unit, 2 audio MMTP payload generation unit, 3 HEVC coding unit (video coding means), 4 video MMTP payload generation unit (video coding means), 5 control information coding unit (control information coding means) ), 6 control MMTP payload generation unit (control information encoding unit), 7 MMTP packet generation unit (multiplexing unit), 11 stream selection unit (bit stream selection unit), 12 MMTP packet analysis unit, 13 control MMTP payload processing unit (Presentation time calculation means), 14 asset separation section, 15 audio MMTP payload processing section, 16 audio ES buffer, 17 audio stream decoding section, 18 audio data buffer, 19 video MMTP payload processing section (video decoding means), 20 HEVCES buffer (Video decoding means), 21 HEVCES decoding (Video decoding means), 22 the decoded picture buffer (video decoding means).

Claims (1)

A decoding device for decoding encoded data of a video signal transmitted by a transmission method defined by MMT (MPEG Media Transport),
Presentation time information indicating the presentation time of the first access unit in the presentation order in the GOP, which is a set of a plurality of access units encoded by the interframe predictive coding method, from the encoded data, and the presentation time information in the presentation order. Control information decoding means for obtaining the information for calculating the presentation time of the leading access unit in the encoding order from the presentation time information indicating the presentation time of the access unit,
The encoded data is obtained by using the presentation time information of the access unit at the beginning in the presentation order and the information for calculating the presentation time of the access unit at the beginning in the encoding order acquired by the control information decoding unit. Decoding means for decoding the video signal included in
Decoding device provided with.

Images