JP5961317B2 - Transmitting apparatus, transmitting method, receiving apparatus, and receiving method - Google Patents

Description

  The present technology relates to a transmission device, a transmission method, a reception device, and a reception method, and more particularly to a transmission device that continuously transmits video streams of a plurality of services.

  In broadcasting, it is conceivable to transmit a plurality of services having different image formats, for example, frame rates. For example, sports programs are provided by 120P service, and others are provided by 60P service.

  Conventionally, for example, in HEVC (High Efficiency Video Coding), temporal direction scalability has been proposed by hierarchically encoding image data of each picture constituting moving image data (see Non-Patent Document 1). On the receiving side, the hierarchy of each picture can be identified based on temporal ID (temporal_id) information inserted in the header of a NAL (Network Abstraction Layer) unit, and selective decoding up to a hierarchy corresponding to the decoding capability is possible. Become.

Gary J. Sullivan, Jens-Rainer Ohm, Woo-Jin Han, Thomas Wiegand, “Overview of the High Efficiency Video Coding (HEVC) Standard” IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECNOROGY, VOL. 22, NO. 12, pp 1649-1668, DECEMBER 2012

  An object of the present technology is to easily avoid the occurrence of display mute at the time of service switching.

The concept of this technology is
Image code that hierarchically encodes the image data of each picture constituting the moving image data and generates an extended stream having encoded image data of the lower layer picture and encoded image data of the higher layer picture Equipped with
When the hierarchical structure of each picture of the basic stream changes, the image encoding unit sets a hierarchy of pictures included in the extended stream as a fixed hierarchy larger than a maximum hierarchy that the picture of the basic stream can take,
A transmission unit for transmitting a multiplexed stream including the basic stream and the extension stream generated by the image encoding unit;
The transmission apparatus further includes an information insertion unit that inserts a level designation value of the bit rate of the basic stream into the multiplexed stream.

Other concepts of this technology are
A basic stream having encoded image data of pictures on the lower layer side and an extended stream having encoded image data of pictures on the higher layer side, generated by hierarchically encoding the image data of each picture constituting the moving image data A receiving unit that receives a multiplexed stream including:
When the hierarchical structure of each picture of the basic stream changes, the hierarchy of pictures included in the extension stream is a fixed hierarchy larger than the maximum hierarchy that the picture of the basic stream can take,
The level designation value of the bit rate of the basic stream is inserted into the multiplexed stream,
Depending on the decoding capability, only the encoded image data of the pictures of each layer included in the basic stream is extracted, or the encoded image data of the pictures of each layer included in the basic stream and the above-mentioned included in the extension stream The receiving apparatus further includes a processing unit that extracts both encoded image data of pictures in a fixed hierarchy and executes decoding processing on the extracted encoded image data of each picture.

According to the present technology, it is possible to easily avoid the occurrence of display mute when switching services.
Note that the effects described here are not necessarily limited, and may be any of the effects described in the present disclosure.

It is a block diagram which shows the structural example of the transmission / reception system as embodiment. It is a figure which shows an example of the case of service switching. It is a figure which shows an example of hierarchical encoding. It is a figure which shows the structure example of the NAL unit header, and the content of the main parameters in the structure example. It is a block diagram which shows the structural example of a transmitter. It is a figure which shows the structural example of a HEVC descriptor. It is a figure which shows the structural example of a seamless switch descriptor. It is a figure which shows the content of the main information in the structural example of a seamless switch descriptor. It is a figure which shows the structural example of a video decoding control descriptor. It is a figure which shows the content of the main information in the structural example of a video decoding control descriptor. It is a figure which shows the structure of a multiplexer roughly. It is a figure which shows the structural example of the transport stream TS in the case of transmitting the video stream (only basic stream is included) of a 60P service. It is a figure which shows the structural example of the transport stream TS in the case of transmitting the video stream (a basic stream and an extended stream are included) of a 120P service. It is a block diagram which shows the structural example of a receiver. It is a figure which shows the structural example of a demultiplexer. It is a figure which shows the operation example of the transmission side (transmission apparatus). It is a figure which shows the operation example of the receiving side (receiving apparatus). It is a figure which shows the other operation example of the transmission side (transmission apparatus). It is a figure which shows the other operation example of the receiving side (receiving apparatus). It is a figure which shows an example of the change of the hierarchical coding structure accompanying the service switching operation. It is a figure which shows another example of the change of the hierarchical coding structure accompanying the service switching operation. It is a figure which shows another example of the change of the hierarchical coding structure accompanying the service switching operation.

Hereinafter, modes for carrying out the invention (hereinafter referred to as “embodiments”) will be described. The description will be given in the following order.
1. Embodiment 2. FIG. Modified example

<1. Embodiment>
[Transmission / reception system]
FIG. 1 shows a configuration example of a transmission / reception system 10 as an embodiment. The transmission / reception system 10 includes a transmission device 100 and a reception device 200.

  The transmission device 100 transmits a transport stream TS as a container on a broadcast wave or a network packet. The transport stream TS includes a plurality of service video streams continuously. In this case, each video stream is composed of first to Mth (M ≦ N) streams among first to N. In this case, a fixed PID (packet identifier) is assigned to each of the first to N streams.

  In the first to Mth streams constituting each video stream, the image data of each picture constituting the moving image data is classified into a plurality of layers, and the image data of the classified pictures of each layer is encoded. At the same time, the plurality of hierarchies are divided into M hierarchies, and each of the divided hierarchies has the encoded image data of the picture. In this case, the hierarchical coding is performed so that the picture layers included in each of the first to Mth streams fall within a mutually independent range assigned in advance.

  FIG. 2 shows an example of a service switching case. FIG. 2 (a) shows switching from a 2160/60 / p video format service to a 2160/120 / p video format service. In this case, switching from one stream to two streams is performed. FIG. 2B shows switching from a 1080/60 / p video format service to a 2160/60 / p video format service. In this case, switching from one stream to one stream is performed.

  FIG. 2C shows switching from a 1080/60 / p video format service to a 2160/120 / p video format service. In this case, switching from one stream to two streams is performed. FIG. 2 (d) shows a switch from a 2160/120 / p video format service to a 1080/60 / p video format service. In this case, switching from 2 streams to 1 stream is performed.

  FIG. 2 (e) shows switching from a 1080/60 / p video format service to a 1080/60 / i video format service. In this case, switching from one stream to one stream is performed. FIG. 2 (f) shows switching from a 3 program service with a video format of 2160/60 / p to a service with a video format of 4320/60 / p. In this case, switching from 3 streams to 1 stream

  FIG. 3 shows an example of hierarchical coding, and each rectangular frame represents a picture. FIG. 3A shows an example in which the highest layer is set to 3, and all of the layers 0 to 3 are set as the hierarchical range of the basic stream. In this example, pictures exist in all the layers from 0 to 3. FIG. 3B is an example in which the highest hierarchy is set to 3, and all of the 0 to 3 hierarchies are set as the hierarchy range of the basic stream. In this example, no picture exists in the third hierarchy, which is a hierarchy gap. FIG. 3C is an example in which the highest hierarchy is set to 3, and all of the 0 to 3 hierarchies are set as the hierarchy range of the basic stream. In this example, no picture exists in the second hierarchy, which is a hierarchy gap.

  FIG. 3D is an example in which the highest layer is 4, in which the layer from 0 to 3 is the basic stream layer range, and the layer 4 is the extended stream layer range. In this example, no picture exists in the third hierarchy, which is a hierarchy gap. FIG. 3 (e) is an example in which the highest layer is 3, in which the layer from 0 to 2 is the basic stream layer range, and the third layer is the extended stream layer range. In this example, pictures exist in all the layers from 0 to 3.

  In hierarchical coding, for example, H.264 is used. H.264 / AVC, H.H. Encoding such as H.265 / HEVC is performed, and the referenced picture is encoded so as to belong to the self hierarchy and / or a hierarchy lower than the self hierarchy. Hierarchy identification information (temporal_id) for identifying the affiliation hierarchy is added to the encoded image data of the picture of each hierarchy for each picture. In the header portion of each picture NAL unit (nal_unit), “nuh_temporal_id_plus1”) representing the layer identification information (temporal_id) is arranged. By adding the layer identification information in this way, the receiving side can know the layer to which each picture belongs.

  FIG. 4A shows a structure example (Syntax) of the NAL unit header, and FIG. 4B shows contents (Semantics) of main parameters in the structure example. In the 1-bit field of “Forbidden_zero_bit”, 0 is essential. The 6-bit field “Nal_unit_type” indicates the NAL unit type. The 6-bit field of “Nuh_layer_id” is assumed to be 0. A 3-bit field of “Nuh_temporal_id_plus1” indicates temporal_id and takes a value (1 to 6) obtained by adding 1.

  In the video stream transmission period of each service, PID (packet identifier) information is inserted into the container layer. In this case, for example, it is inserted in one of the following first and second PID information insertion modes. This PID information is inserted under a program map table (PMT), for example.

“First PID information insertion mode”
In the first PID information insertion mode, in the transmission period of a video stream of a certain service, the PID information of each stream constituting the video stream is always inserted, and the transmission period of the video stream of the next service is started. PID information of each stream constituting this video stream is inserted at the immediately preceding timing.

“Second PID information insertion mode”
In the second PID information insertion mode, the PID information of each of the first to Nth streams is inserted in the transmission period of the video stream of each service.

  Further, in the transmission period of the video stream of each service, the hierarchical range information is inserted into the container layer. This hierarchical range information is inserted, for example, under a program map table (PMT). In this case, in the transmission period of the video stream of a certain service, the timing immediately before the transmission of the video stream of the next service is always started after the hierarchical range information of the pictures of each stream constituting the video stream is inserted. The layer range information of the pictures of each stream constituting this video stream is inserted.

  Further, switching information is inserted into the container layer during the transmission period of the video stream of each service. This switching information includes, for example, frame rate information of encoded image data included in the video stream of the next service, stream configuration information, and the like. This switching information is inserted, for example, under the program map table (PMT).

  The receiving device 200 receives the above-described transport stream TS transmitted from the transmitting device 100 on broadcast waves or net packets. In this transport stream TS, video streams of a plurality of services are continuously included. Each video stream is composed of first to Mth (M ≦ N) streams among first to Nth streams. Here, a fixed PID (packet identifier) is assigned to each of the first to N streams.

  The receiving apparatus 200 recognizes the PID of each stream constituting the video stream included in the transport stream TS from the PID (packet identifier) information inserted in the container layer. The receiving apparatus 200 processes each stream constituting the video stream by filtering based on the PID assigned thereto. The receiving apparatus 200 sequentially displays images based on the video streams of the services that are continuously included in the transport stream TS.

  In this case, since a fixed PID (packet identifier) is assigned to each of the first to N streams, even when the service is switched, it is not necessary to change the setting of the filter for extracting each stream. Therefore, the occurrence of display mute is suppressed and seamless display is performed.

  The receiving apparatus 200 recognizes the hierarchical range of the pictures of each stream constituting the video stream included in the transport stream TS from the hierarchical range information inserted in the container layer. In this case, since the hierarchy of pictures to be included in each of the first to Mth streams is hierarchically encoded so as to fall within a pre-assigned range, which stream includes a predetermined hierarchy. Or, it is avoided that the system layer and the video layer are inconsistent.

  The receiving apparatus 200 recognizes the frame rate information of the encoded image data included in the video stream of the next service from the switching information inserted in the container layer before the service switching.

  In this embodiment, N = 2 and each video stream is composed of only a basic stream (Base stream) or a basic stream (Base stream) and an enhanced stream (Enhanced stream). Here, in the case of the 60P service, the video stream is configured by only the basic stream, and in the case of the 120P service, the video stream is configured by the basic stream and the extension stream.

  In this embodiment, encoding is performed so that the number of picture layers included in the extension stream is one. In addition, in the transmission period of the video stream composed only of the basic stream, any one of the following first and second aspects is adopted. In the first aspect, only the hierarchical range information of the basic stream is inserted. In the second aspect, in addition to the layer range information of the basic stream, information on the layer range that can be taken by the extension stream is inserted.

  In this case, for example, the values of the minimum (mix) and the maximum (max) are different from each other, which means that there is actually no extension stream. That is, in this embodiment, encoding is performed so that the picture stream included in the extension stream is one. When the extension stream exists, the minimum (mix) and maximum (max) values are Because it becomes the same.

"Configuration of Transmitter"
FIG. 5 shows a configuration example of the transmission device 100. The transmission apparatus 100 includes a CPU (Central Processing Unit) 101, encoders 102A and 102B, compressed data buffers (cpb: coded picture buffer) 103A and 103B, a multiplexer 104, and a transmission unit 105. The CPU 101 is a control unit and controls the operation of each unit of the transmission device 100.

  The encoder 102A inputs uncompressed moving image data VDA having a frame frequency of 60 Hz. H.264 / AVC, H.H. Encoding such as H.265 / HEVC is performed. At this time, the encoder 102A classifies the image data of each picture constituting the moving image data VDA into a plurality of hierarchies, encodes the image data of the classified pictures of each hierarchy, and encodes the pictures of each hierarchy. A video stream related to a 60P service having image data is generated. The compressed data buffer (cpb) 103A temporarily stores this video stream.

  This video stream includes only the basic stream. That is, the encoder 102A generates a basic stream having encoded image data of pictures of all layers obtained by layer encoding. Note that hierarchical encoding is performed so that the hierarchical range of pictures included in the basic stream is a previously allocated range. For example, when the hierarchical range of pictures included in the basic stream is a hierarchy of 0 to 3, for example, as shown in FIGS. 3A to 3C, hierarchical encoding is performed so that the highest hierarchy is 3 or less. Is done.

  The encoder 102B inputs uncompressed moving image data VDB having a frame frequency of 120 Hz. H.264 / AVC, H.H. Encoding such as H.265 / HEVC is performed. At this time, the encoder 102B classifies the image data of each picture constituting the moving image data VDB into a plurality of layers, encodes the image data of the classified pictures of each layer, and encodes the pictures of each layer. A video stream related to a 120P service having image data is generated. The compressed data buffer (cpb) 103B temporarily stores this video stream.

  This video stream includes a basic stream and an extension stream. That is, the encoder 102B bisects a plurality of hierarchies and generates a basic stream having encoded image data of pictures on the lower hierarchy side and an extension stream having encoded image data of pictures on the higher hierarchy side.

  Note that the hierarchical encoding of pictures included in the basic stream and the extension stream is hierarchically encoded so as to be a pre-assigned range, and the encoding of the pictures included in the extended stream is one. For example, when the hierarchical range of pictures included in the basic stream is a hierarchy of 0 to 3 and the hierarchical range of pictures included in the extension stream is a hierarchy of 4 to 5, for example, as illustrated in FIG. The hierarchy of pictures included in the basic stream is 3 or less, and the hierarchy of pictures included in the extension stream is 4.

  When performing the 60P service, the multiplexer 104 reads out the video stream stored in the compressed data buffer 103A, converts it into a PES packet, further multiplexes it into a transport packet, and obtains a transport stream TS as a multiplexed stream. As described above, the transport stream TS includes only the basic stream. The multiplexer 104 assigns a fixed PID (= PID_1) to this basic stream.

  Further, when performing the service of 120P, the multiplexer 104 reads out the video stream stored in the compressed data buffer 103B, converts it into a PES packet, further converts it into a transport packet, multiplexes, and multiplexes the transport stream TS as a multiplexed stream. obtain. As described above, the transport stream TS includes a basic stream and an extension stream. The multiplexer 104 assigns a fixed PID (= PID_1) to the basic stream and assigns a fixed PID (= PID_2) to the extension stream.

  The transmitting unit 105 transmits the transport stream TS obtained by the multiplexer 104 to a receiving device 200 on a broadcast wave or a net packet. In this case, for example, the transport stream TS related to the 60P service is transmitted, and then the transport stream TS related to the 120P service is transmitted. Alternatively, the transport stream TS related to the 120P service is transmitted, and then the transport stream TS related to the 60P service is transmitted.

  The multiplexer 140 inserts PID information under the program map table (PMT) during the transmission period of the video stream of each service. In this case, the multiplexer 140 inserts the PID information in the following first PID information insertion mode or second PID information insertion mode.

  That is, in the first PID information insertion mode, in the transmission period of the video stream of a certain service, the PID information of each stream constituting the video stream is always inserted, and the transmission period of the video stream of the next service starts. The PID information of each stream constituting this video stream is inserted at the timing immediately before being performed. In the second PID information insertion mode, PID information of both the basic stream and the extended stream is inserted in the transmission period of the video stream of each service.

  Further, the multiplexer 104 inserts layer range information under the program map table (PMT) during the transmission period of the video stream of each service. In this case, in the transmission period of the video stream of a certain service, the timing immediately before the transmission period of the video stream of the next service is always inserted by inserting the hierarchical range information of the pictures of each stream constituting the video stream. The layer range information of pictures of each stream constituting this video stream is inserted.

  Here, the multiplexer 104 inserts the layer range information in the following first mode or second mode during the transmission period of the video stream composed of only the basic stream. That is, in the first mode, only the hierarchical range information of the basic stream is inserted. Also, in the second aspect, information on the hierarchical range that can be taken by the extended stream is inserted together with the hierarchical range information of the basic stream.

  As described above, there is one picture layer included in the extension stream, but the information of the hierarchy range that can be taken by this extension stream includes, for example, a plurality of hierarchies. Thereby, the information on the hierarchical range that can be taken by the extended stream literally indicates the hierarchical range that can be taken by the extended stream, and suggests that the extended stream does not actually exist.

  For insertion of this hierarchical range information, the fields of “temporal_id_min” and “temporal_id_max” of the existing HEVC descriptor (HEVC descriptor) are used. FIG. 6 shows a structural example (Syntax) of the HEVC descriptor (HEVC_descriptor). The 8-bit field of “descriptor_tag” indicates a descriptor type, and here indicates that it is a HEVC descriptor. The 8-bit field of “descriptor_length” indicates the length (size) of the descriptor, and indicates the number of subsequent bytes as the length of the descriptor.

  The 8-bit field of “level_idc” indicates a bit rate level designation value. When “temporal_layer_subset_flag = 1”, a 5-bit field of “temporal_id_min” and a 5-bit field of “temporal_id_max” exist. “Temporal_id_min” indicates the value of temporal_id of the lowest hierarchy of the hierarchically encoded data included in the corresponding video stream. “Temporal_id_max” indicates the value of temporal_id of the highest hierarchy of the hierarchically encoded data included in the corresponding video stream.

  Further, the multiplexer 140 inserts the switching information under the program map table (PMT) at the timing just before the transmission period of the video stream of the next service is started in the transmission period of the video stream of each service. To do. In order to insert the switching information, a newly defined seamless switch descriptor (Seamless_switch descriptor) or an existing video decode control descriptor (Video_decode_control descriptor) is used.

  FIG. 7 shows a structural example (Syntax) of the seamless switch descriptor. FIG. 8 shows the contents (Semantics) of main information in the structural example. An 8-bit field of “descriptor_tag” indicates a descriptor type. Here, it is shown that it is a seamless switch descriptor. The 8-bit field of “descriptor_length” indicates the length (size) of the descriptor, and indicates the number of subsequent bytes as the length of the descriptor.

  The 1-bit field of “EOS_flag” is a flag indicating that “end_of_seq” is encoded at the end of the stream. “1” indicates encoding. “0” indicates that encoding is not performed. A 3-bit field of “number_of_streams” indicates the number of service streams after switching. A 4-bit field of “frame_rate” indicates the frame frequency of the service stream after switching. For example, “1001” indicates 60 Hz, and “1100” indicates 120 Hz.

  The 4-bit field of “spatial_resolution” indicates the spatial resolution of the service stream after switching. For example, “0001” indicates 720 (h) * 480 (v), “0010” indicates 1280 (h) * 720 (v), “0011” indicates 1920 (h) * 1080 (v), “0100” indicates 3840 (h) * 2160 (v), and “0101” indicates 7680 (h) * 4320 (v). A 1-bit field of “scanning_format” indicates a frame structure. “1” indicates progressive, and “0” indicates interlace.

  FIG. 9 shows a structural example (Syntax) of the video decode control descriptor. FIG. 10 shows the contents (Semantics) of main information in the structural example. An 8-bit field of “descriptor_tag” indicates a descriptor type. Here, the video decoding control descriptor is indicated. The 8-bit field of “descriptor_length” indicates the length (size) of the descriptor, and indicates the number of subsequent bytes as the length of the descriptor.

  A 1-bit field of “sequence_end_code_flag” is a flag indicating that “end_of_seq” is encoded at the end of the stream. “1” indicates encoding. “0” indicates that encoding is not performed. A 4-bit field of “video_encode_format” indicates the format of the service stream after switching. For example, “0000” indicates 1080p (2K), “0111” indicates 2160p (4K), and “1000” indicates 4320p (8K).

  A 1-bit field of “frame_rate_type” is a newly defined field, indicating the frame frequency type of the stream after switching, “1” is a conventional frame rate of 60 Hz or less, and “0” is 120 Hz or more. Indicates to configure. A 1-bit field of “stream_not_extended_flag” is a newly defined field, which indicates that an extended stream does not exist in the stream after switching, “1” does not include an extended stream of another PID, and “0” indicates a different PID. It shows that the extension stream is multiplexed and combined to form 120p.

  FIG. 11 schematically shows the configuration of the multiplexer 104. The multiplexer 104 includes a section information (including descriptor) generation unit 141, a null packet generation unit 142, a selector 143, a PID allocation unit 144, and a TS multiplexing unit 145. The selector 143 is generated in the 60P service video stream (basic stream), 120P service video stream (basic stream, extension stream), other service streams, and section information generator 141 input from the outside. Section information and a null packet generated by the null packet generator 142 are input and either one is selectively extracted.

  The PID assignment unit 144 assigns a PID to each signal selectively extracted by the selector 143, that is, a video stream, section information, a null packet, and the like. The TS multiplexing unit 145 multiplexes each signal to obtain a transport stream TS.

[Configuration of transport stream TS]
FIG. 12 illustrates a configuration example of the transport stream TS when a 60P service video stream (including only a basic stream) is transmitted. In this configuration example, the PES packet “video PES1” of the basic stream identified by PID1 exists. Here, PID1 is a PID fixed to the basic stream.

  NAL units such as VPS, SPS, PPS, SLICE, and SEI exist in the encoded image data of each picture. As described above, the layer identification information (“nuh_temporal_id_plus1” meaning temporal_id) of the picture is arranged in the header of the NAL unit. In the SPS, “general_level_idc” that is a level designation value of the bitstream is inserted.

  In addition, the transport stream TS includes a PMT (Program Map Table) as PSI (Program Specific Information). PSI is information describing to which program each elementary stream included in the transport stream belongs. In the PMT, there is a program loop that describes information related to the entire program. The PMT includes an elementary loop having information related to each elementary stream. In this configuration example, there is a video elementary loop (video ES1 loop) corresponding to the basic stream.

  In the video elementary loop (video ES1 loop), information such as a stream type and a PID (packet identifier) is arranged corresponding to the basic stream (video PES1), and information related to the video stream is stored. A descriptor to be described is also arranged. The value of “Stream_type” of the basic stream is set to “0x24”, and the PID information indicates PID1 given to the PES packet “video PES1” of the basic stream as described above.

  Further, the above-described HEVC descriptor (HEVC descriptor) and seamless switch descriptor (Seamless_switch descriptor) are inserted as descriptors arranged in the video elementary loop (video ES1 loop). When a newly defined video decode control descriptor (Video_decode_control descriptor) is used for the “frame_rate” field, it is not necessary to insert a seamless switch descriptor.

  In the illustrated example, “level_idc” is set to “level 5.1” in the HEVC descriptor. Also, “temporal_id_min = 0” and “temporal_id_max = 3” are set, indicating that the hierarchical range of the basic stream is a hierarchy from 0 to 3.

  In the first mode, only the video elementary loop (video ES1 loop) corresponding to the basic stream exists under the PMT, but in the second mode, it corresponds to the extension stream illustrated by the broken line. There is also a video elementary loop (video ES2 loop).

  In this video elementary loop (video ES2 loop), information such as a stream type and a PID (packet identifier) is arranged corresponding to an extension stream (video PES2) that does not actually exist, and the video stream A descriptor describing information related to the is also arranged. The value of “Stream_type” of the extension stream is set to “0x25”, and the PID information indicates PID2 given to the PES packet “video PES2” of the extension stream.

  In addition, the above-described HEVC descriptor (HEVC descriptor) and seamless switch descriptor (Seamless_switch descriptor) are inserted as descriptors arranged in the video elementary loop. When a newly defined video decode control descriptor (Video_decode_control descriptor) is used for the “frame_rate” field, it is not necessary to insert a seamless switch descriptor.

  In the illustrated example, “level_idc” is set to “level 5.2” in the HEVC descriptor. Also, “temporal_id_min = 5” and “temporal_id_max = 6” are set, indicating that the hierarchical range that the extension stream can take is the hierarchy of 5 and 6, and there is actually no extension stream. Is shown.

  Note that the PMT included in the transport stream TS when transmitting a 60P service video stream (including only a basic stream) is always the content shown in the figure, but transmission of the next service video stream is started. At the timing immediately before the update, the content is updated to a new PMT having contents corresponding to the video stream.

  FIG. 13 illustrates a configuration example of the transport stream TS when a 120P service video stream (including a basic stream and an extended stream) is transmitted. In this configuration example, the PES packet “video PES1” of the basic stream identified by PID1 and the PES packet “video PES2” of the extension stream identified by PID2 exist. Here, “stream_id” is the same service (broadcast channel), PID1 is a PID fixed to the basic stream, and PID2 is a PID fixed to the extension stream.

  The encoded image data of each picture of the basic stream includes NAL units such as VPS, SPS, PPS, SLICE, and SEI. In the header of the NAL unit, layer identification information (“nuh_temporal_id_plus1” meaning temporal_id) of the picture is arranged. In the SPS, “general_level_idc” that is a level designation value of the bitstream is inserted. Also, in SPS, pictures belonging to each layer indicated by “temporal_id” are bundled as sublayers (sub_layer), and “sublayer_level_presented_flag” is set to “1”, which is a bit rate level designation value for each sublayer. “Sublayer_level_idc” is inserted.

  On the other hand, NAL units such as PPS and SLICE exist in the encoded image data of each picture of the extension stream. In the header of the NAL unit, layer identification information (“nuh_temporal_id_plus1” meaning temporal_id) of the picture is arranged.

  In addition, the transport stream TS includes a PMT (Program Map Table) as PSI (Program Specific Information). This PSI is information describing to which program each elementary stream included in the transport stream belongs.

  In the PMT, there is a program loop that describes information related to the entire program. The PMT includes an elementary loop having information related to each elementary stream. In this configuration example, there are two video elementary loops (video ES1 loop, video ES2 loop).

  In each video elementary loop, information such as a stream type and a packet identifier (PID) is arranged corresponding to the video stream (video PES1, video PES2), and information related to the video stream is described. A descriptor is also placed. Although detailed description of the video elementary loop (video ES1 loop) is omitted, it is the same as the TS configuration in FIG.

  In the video elementary loop (video ES2 loop), information such as stream type and PID (packet identifier) is arranged corresponding to the extension stream (video PES2), and information related to the video stream is described. A descriptor is also arranged. The value of “Stream_type” of the extension stream is set to “0x25”, and the PID information indicates PID2 given to the PES packet “video PES2” of the extension stream.

  In addition, the above-described HEVC descriptor (HEVC descriptor) and seamless switch descriptor (Seamless_switch descriptor) are inserted as descriptors arranged in the video elementary loop. When a newly defined video decode control descriptor (Video_decode_control descriptor) is used for the “frame_rate” field, it is not necessary to insert a seamless switch descriptor. In the illustrated example, “level_idc” is set to “level 5.2” in the HEVC descriptor. Further, “temporal_id_min = 5” and “temporal_id_max = 5” are set, which indicates that the hierarchy of pictures included in the extension stream is 5.

  Note that the PMT included in the transport stream TS in the case of transmitting a 120P service video stream (including only the basic stream) is always shown in the figure, but transmission of the next service video stream is started. At the timing immediately before the update, the content is updated to a new PMT having contents corresponding to the video stream.

  The operation of the transmission apparatus 100 shown in FIG. 5 will be briefly described. The encoder 102A receives uncompressed moving image data VDA having a frame frequency of 60 Hz. In this encoder 102A, for example, H.P. H.264 / AVC, H.H. Coding such as H.265 / HEVC is performed. At this time, the encoder 102A classifies the image data of each picture constituting the moving image data VDA into a plurality of hierarchies, encodes the image data of the classified pictures of each hierarchy, and codes the pictures of the respective hierarchies. A video stream related to a 60P service having converted image data is generated.

  This video stream includes only the basic stream. That is, the encoder 102A generates a basic stream having encoded image data of pictures of all layers obtained by layer encoding. Note that hierarchical encoding is performed so that the hierarchical range of pictures included in the basic stream is a previously allocated range. The video stream (basic stream) generated by the encoder 102A and including the encoded data of the pictures of each layer is supplied to the compressed data buffer (cpb) 103A and temporarily stored.

  Also, uncompressed moving image data VDB having a frame frequency of 120 Hz is input to the encoder 102B. In this encoder 102B, for example, the H.264. H.264 / AVC, H.H. Coding such as H.265 / HEVC is performed. At this time, the encoder 102B classifies the image data of each picture constituting the moving image data VDB into a plurality of hierarchies, encodes the image data of the classified pictures of each hierarchy, and codes the pictures of the respective hierarchies. A video stream related to a 120P service having converted image data is generated.

  This video stream includes a basic stream and an extension stream. That is, in the encoder 102B, a plurality of hierarchies are divided into two, and a basic stream having encoded image data of pictures on the lower hierarchy side and an extension stream having encoded image data of pictures on the higher hierarchy side are generated. Note that the hierarchical encoding of pictures included in the basic stream and the extension stream is hierarchically encoded so as to be a pre-assigned range, and the encoding of the pictures included in the extended stream is one. The video stream (basic stream and extended stream) generated by the encoder 102B and including the encoded data of the pictures of each layer is supplied to the compressed data buffer (cpb) 103B and temporarily stored.

  In the multiplexer 104, when the 60P service is performed, the video stream stored in the compressed data buffer 103A is read out, converted into a PES packet, further multiplexed into a transport packet, and a transport stream as a multiplexed stream. TS is obtained. This transport stream TS includes only the basic stream. In the multiplexer 104, a fixed PID (= PID_1) is assigned to this basic stream.

  Further, in the case of performing the 120P service, the multiplexer 104 reads out the video stream stored in the compressed data buffer 103B, converts it into a PES packet, further multiplexes it into a transport packet, and transmits it as a multiplexed stream. A port stream TS is obtained. The transport stream TS includes a basic stream and an extension stream. In the multiplexer 104, a fixed PID (= PID_1) is assigned to the basic stream, and a fixed PID (= PID_2) is assigned to the extension stream.

  Further, in the multiplexer 104, PID information is inserted under the program map table (PMT) during the transmission period of the video stream of each service. Here, in the first PID information insertion mode, in the transmission period of a video stream of a certain service, the PID information of each stream constituting the video stream is always inserted, and the transmission period of the video stream of the next service is PID information of each stream constituting this video stream is inserted at a timing immediately before the start. In the second PID information insertion mode, PID information of both the basic stream and the extended stream is inserted during the transmission period of the video stream of each service.

  Further, in the multiplexer 104, the hierarchical range information is inserted under the program map table (PMT) during the transmission period of the video stream of each service. In this case, in the transmission period of the video stream of a certain service, the timing immediately before the transmission of the video stream of the next service is always started after the hierarchical range information of the pictures of each stream constituting the video stream is inserted. The layer range information of the pictures of each stream constituting this video stream is inserted.

  Here, in the multiplexer 104, only the layer range information of the basic stream is inserted in the first mode and the layer range information of the basic stream is extended in the second mode in the transmission period of the video stream composed of only the basic stream. Information on the hierarchical range that can be taken by the stream is also inserted. The hierarchy of pictures included in the extension stream is one, but the information of the hierarchy range that can be taken by this extension stream includes, for example, a plurality of hierarchies.

  Further, in the multiplexer 140, the switching information is inserted under the program map table (PMT) at least immediately before the transmission period of the video stream of the next service is started in the transmission period of the video stream of each service. Is done. For the insertion of this switching information, a newly defined seamless switch descriptor (Seamless_switch descriptor) or an existing video decode control descriptor (Video_decode_control descriptor) is used.

  In the transmission unit 105, the transport stream TS obtained by the multiplexer 104 is transmitted to the reception device 200 on a broadcast wave or a net packet. In this case, for example, the transport stream TS related to the 60P service is transmitted, and then the transport stream TS related to the 120P service is transmitted. Alternatively, the transport stream TS related to the 120P service is transmitted, and then the transport stream TS related to the 60P service is transmitted.

  In the above description, the encoder 102A generates a video stream (basic stream) related to the 60P service, and the encoder 102B generates a video stream (basic stream, extended stream) related to the 120P service, and the multiplexer 104 switches between these. I explained that. However, the same effect can be given to the encoder 102B. In this case, the encoder 102B inputs uncompressed moving image data VDB having a frame frequency of 120 Hz, outputs a video stream (basic stream) related to the 60P service, and a video stream (basic stream, extended) related to the 120P service. (Stream) output and switching output.

"Receiver configuration"
FIG. 14 illustrates a configuration example of the receiving device 200. The receiving apparatus 200 includes a CPU (Central Processing Unit) 201, a receiving unit 202, a demultiplexer 203, and a compressed data buffer (cpb: coded picture buffer) 204. The receiving apparatus 200 includes a decoder 205, an uncompressed data buffer (dpb: decoded picture buffer) 206, a post processing unit 207, and a display unit 208. The CPU 201 constitutes a control unit and controls the operation of each unit of the receiving device 200.

  The reception unit 202 receives the transport stream TS transmitted from the transmission device 100 on broadcast waves or net packets. The demultiplexer 203 extracts a stream constituting the video stream included in the transport stream TS by filtering with a PID filter, and sends the stream to a compressed data buffer (cpb: coded picture buffer) 204.

  In this case, a fixed PID is assigned to each of the basic stream and the extension stream. Therefore, for example, even when the service is switched from the 60p service to the 120p service or vice versa, it is not necessary to change the setting of the filter for extracting each stream. Display is possible.

  FIG. 15 shows a configuration example of the demultiplexer 203. The demultiplexer 203 includes a PID filter 231, multiplexing buffers 232 — 0 to 232 — n, 232 — null, 232 — c, a section filter 233, and a PMT analysis unit 234.

  The PID filter 231 passes section data and null packets included in the transport stream TS based on the PID. In the illustrated example, the PID value of the section data is PID_c, and the PID of the null packet is PID_null. The null packet does not have a unique PID value and may be transmitted so as to be inserted into a video PID stream. This makes it possible to detect that a null packet is detected in the multiplexing buffer of the receiver and to determine that switching occurs. Further, the PID filter 231 passes TS packets corresponding to a program number (program number) corresponding to the broadcast service channel included in the transport stream TS based on the set PID value. In the illustrated example, PID values of TS packets that can be set are PID_0 to PID_n.

  The multiplexing buffers 232_0 to 232_n, 232_null, and 232_c temporarily store the TS packets, section data, and null packets that have passed through the PID filter 231, respectively. That is, in the demultiplexer 203, the multiplexing buffer is managed for each PID value. The section filter 233 extracts a program map table (PMT) from the section data stored in the multiplexing buffer 232_c based on the PID value.

  The PMT analysis unit 234 analyzes the PMT extracted by the section filter 233, and sets the PID value of the TS packet to be passed to the PID filter 231 based on the analysis result. For example, when the transport stream TS is a 60P service, for example, “101” that is a fixed PID value assigned to the basic stream is set as PID_0. Also, for example, when the transport stream TS is a 120P service, for example, “101” which is a fixed PID value assigned to the basic stream as PID_0 is set, and is given to the extension stream as PID_1. A fixed PID value “102” is set.

  The demultiplexer 203 transfers TS packets stored for each PID value in the multiplexing buffers 232_0 to 232_n to the compressed data buffer 204 in accordance with the decoding capability of the decoder 205. For example, when the decoder 205 is a 60p decoder, the TS packets related to the basic stream stored in the multiplexing buffer 232_0 are transferred to the compressed data buffer 204. For example, when the decoder 205 is a 120p decoder, TS packets related to the basic stream stored in the multiplexing buffer 232_0 and TS packets related to the extension stream stored in the multiplexing buffer 232_1 are stored in the compressed data buffer 204. Forward.

  In the above description, for example, when the transport stream TS is a 120P service, the PID filter 231 passes the PID_0 and PID_1 TS packets. However, when the decoder 205 is a 60p decoder, the PID filter 231 It is also possible to pass only the PID_0 TS packet.

  Returning to FIG. 14, the compressed data buffer (cpb) 204 temporarily stores the TS packets transferred from the demultiplexer 203, and thus the encoded image data of each picture. The decoder 205 reads out and decodes the encoded image data of each picture stored in the compressed data buffer 204 at a decoding timing given by a DTS (Decoding Time stamp) of the picture, and decodes the uncompressed data buffer (dpb ) 206.

  An uncompressed data buffer (dpb) 206 temporarily stores the image data of each picture decoded by the decoder 205. The post processing unit 207 adjusts the frame rate of the image data of each picture sequentially read at the display timing given by the PTS (Presentation Time stamp) from the uncompressed data buffer (dpb) 206 to the display capability. Process.

  For example, when the frame rate of the image data of each picture after decoding is 60 fps and the display capability is 120 fps, the post processing unit 207 has a resolution in the time direction twice that of the image data of each picture after decoding. Interpolation processing is performed so that the image data is 120 fps and sent to the display unit 208.

  The display unit 208 includes, for example, an LCD (Liquid Crystal Display), an organic EL (Organic Electro-Luminescence) panel, and the like. The display unit 208 may be an external device connected to the receiving device 200.

  The operation of the receiving apparatus 200 shown in FIG. 14 will be briefly described. The reception unit 202 receives the transport stream TS transmitted from the transmission device 100 on broadcast waves or net packets. This transport stream TS is sent to the demultiplexer 203. The demultiplexer 203 extracts a TS packet corresponding to the service from the transport stream TS based on the PID information included in the PMT. This TS packet is sent to the compressed data buffer (cpb) 204 and temporarily accumulated.

  For example, when the transport stream TS is a 60p service, TS packets related to the basic stream are extracted and transferred to the compressed data buffer 204. Also, for example, when the transport stream TS is a 120p service, when the decoder 205 is a 60p decoder, TS packets related to the basic stream are extracted and transferred to the compressed data buffer 204, and the decoder 205 is a 120p decoder. Sometimes, TS packets related to both the basic stream and the extension stream are taken out and transferred to the compressed data buffer 204.

  In the decoder 205, the encoded image data of each picture stored in the compressed data buffer 204 is decoded at the decoding timing of the picture, sent to the uncompressed data buffer (dpb) 206, and temporarily stored. The The image data of each picture sequentially read from the uncompressed data buffer (dpb) 206 at the display timing is sent to the post processing unit 207. In the post processing unit 207, interpolation or sub-sampling is performed on the image data of each picture so that the frame rate matches the display capability. The image data of each picture processed by the post processing unit 207 is supplied to the display unit 208, and a moving image is displayed.

  Next, switching operation from the 4K 60p program to the 4K 120p program in the transmission / reception system 10 shown in FIG. 1 will be described. FIG. 16 illustrates an operation example of the transmission side, that is, the transmission apparatus 100. In the 60p service transmission period, the encoder 102A generates a video stream related to the 60p service. This video stream includes only the basic stream to which, for example, “101” is assigned as the PID value.

  In the 60p service transmission period, the multiplexer 104 converts the basic stream into a PES packet, further transports the packet and multiplexes it to obtain a transport stream TS as a multiplexed stream, and transmits a 4K 60p program. It becomes a stream.

  In this 60p service transmission period, a PMT indicated by “old PMT” in the figure is always inserted and transmitted in the container layer. This PMT includes information on “Service_id” and “Version number”, and also includes information on “Elementary_PID” and “Stream_type” corresponding to the basic stream.

  Also, in this 60p service transmission period, at the timing immediately before the 120p service transmission period is started, for example, one second before the end of the 60p service transmission period, the PMT indicated by “new PMT” in the figure is displayed. Is inserted and sent. This PMT includes information on “Service_id” and “Version number” as in the “old PMT”. “Version number” is changed from “V0” to “V0 + 1”, indicating that it is changed to “new PMT”.

  Further, this PMT includes “Elementary_PID” and “Stream_type” information corresponding to the basic stream, and also includes “Elementary_PID” and “Stream_type” information corresponding to the extension stream. Further, the PMT includes a seamless switch descriptor (Seamless_switch descriptor) in which switching information is described.

  Here, “EOS_flag = 1” is set, indicating that “end_of_seq” is encoded. Further, “number_of_streams = 2” is set, indicating that the number of service streams after switching is two. Further, “frame_rate = 1100 (120 Hz)” is set, indicating that the frame frequency of the service stream after switching is 120 Hz.

  When the 60p service transmission period ends, the service transmission period switches to 120p. In the 120p service transmission period, the encoder 102B generates a video stream related to the 120p service. This video stream includes a basic stream to which “101” is assigned as a PID value and an extension stream to which “102” is assigned as a PID value, for example.

  In this 120p service transmission period, the multiplexer 104 converts the basic stream and the extension stream into PES packets, further multiplexes them into transport packets, and obtains a transport stream TS as a multiplexed stream. 4K 120p It becomes the transmission stream of the program. Note that a gap period in which null packets are transmitted is provided between the transmission stream of the 4K 60p program and the transmission stream of the 4K 120p program.

  FIG. 17 shows an operation example of the receiving side, that is, the receiving apparatus 200. In the 60p service transmission period, a video stream of 4K 60 programs is output from the demultiplexer 203. The decoder 205 outputs 4K 60 program image data regardless of whether it is a 60p decoder or a 120p decoder.

  In this 60p service transmission period, the PMT indicated by “old PMT” is always acquired from the container layer, and “new PMT” is acquired at the timing immediately before the 120p service transmission period starts. By setting the PID value of the extension stream included in the “new PMT” in the PID filter, the demultiplexer 203 can output a video stream of 4K 120 programs.

  When the 60p service transmission period ends, the service transmission period switches to 120p. In this case, the output of the demultiplexer 203 changes from a 4K 60p program video stream (basic stream only) to a 4K 120p program video stream (basic stream and extension stream) through a gap period including null packets. The decoder 205 outputs image data of 4K 60 programs when the decoder is a 60p decoder, and outputs image data of 4K 120 programs when the decoder 205 is a 120p decoder.

  The operation of the demultiplexer 203 in switching from the 60p service transmission period to the 120p service transmission period will be further described. In the 60p service transmission period, only the stream (basic stream) of PID_0 (101) passes through the PID filter 231 and is accumulated in the multiplexing buffer 232_0.

  One second before the end of the 60p service transmission period, the section filter 233 extracts “new PMT”. This “new PMT” is analyzed by the PMT analysis unit 234, and the PID value of the stream to be passed through the PID filter 231 as well as “Stream_type” and “Descriptor” are detected, and the PID value is set to the PID filter 231. Is called. As a result, the PID filter 231 can pass the stream (enhanced stream) of PID_1 (102) together with the stream (basic stream) of PID_0 (101) in the subsequent 120p service transmission period.

  After switching to the 120p service transmission period, the stream (basic stream) of PID_0 (101) passes through the PID filter 231 and is accumulated in the multiplexing buffer 232_0, and the stream (extended stream) of PID_0 (102) is stored in the PID filter. 231 is accumulated in the multiplexing buffer 232_1. In addition, when the “EOS_flag” of the seamless switch descriptor of “New PMT” is “1”, switching regarding the video format and the number of service streams may occur when the EOS of the video stream is determined. Determined.

  In the operation examples shown in FIGS. 16 and 17 described above, “new” inserted into the container layer at the timing immediately before the 120p service transmission period starts, for example, one second before the end of the 60p service transmission period. An example is shown in which the PID value of the extension stream is notified by “PMT” (first PID insertion form). However, it is also possible to reserve the PID value of the extension stream by including the PID value of the extension stream in the “old PMT” that is always inserted in the container layer in the 60p service transmission period (second PID insertion form). . FIG. 18 and FIG. 19 show operation examples on the transmission side and the reception side in that case.

  Next, in the transmission / reception system 10 shown in FIG. 1, a change in the hierarchical coding structure accompanying the service switching operation will be described. FIG. 20 shows an example of a change in the hierarchical coding structure accompanying the service switching operation. In this example, layers 0 to 3 are assigned to the basic stream, and layers 4 and 5 are assigned to the extension stream.

  In this example, the video stream of the 60p service includes encoded image data of each picture subjected to hierarchical encoding. In this case, there are 0 to 3 picture layers. The 60p service video stream includes only the basic stream. This basic stream includes pictures of all layers from 0 to 3. In this case, “general_level_idc” is set to “level 5.1” and “sps_max_sublayer_minus1 = 3” in the SPS of the basic stream.

  In this example, the 120p service video stream includes encoded image data of each picture that has been hierarchically encoded. In this case, there are 0 to 4 picture layers. The 120p service video stream includes a basic stream and an extended stream. The basic stream includes pictures in layers 0 to 3, and the extended stream includes pictures in layer 4. In this case, “general_level_idc” is set to “level5.2” and “sps_max_sublayer_minus1 = 4” in the SPS (Sequence Parameter Set) of the basic stream.

  At the timing immediately before the 60p service transmission period starts, a PMT having information on the video stream of the 60p service is inserted into the container layer. In the HEVC descriptor (HEVC descriptor) included in this PMT, “level_idc” is set to “level 5.1”, and further, “temporal_id_min = 0” and “temporal_id_max = 3” are set.

  Similarly, in the 60p service transmission period, a PMT having information on the video stream of the 120p service is inserted into the container layer at a timing immediately before the 120p service is started. In the HEVC descriptor related to the basic stream included in the PMT, “level_idc” is set to “level 5.1”, and further, “temporal_id_min = 0” and “temporal_id_max = 3” are set. In the HEVC descriptor related to the extension stream included in this PMT, “level_idc” is set to “level 5.2”, and further, “temporal_id_min = 4” and “temporal_id_max = 4” are set.

  In this case, in the period T in which the 60p service transmission period ends from the insertion of the PMT having the 120p service video stream information, the video layer information is the 60p service video stream information, whereas the system layer information The information becomes information of a 120p service video stream. However, in this example, the range of the “temporal_id” value of the video matches the range of the “temporal_id” value of the HEVC descriptor. This is because hierarchical coding is performed so that the video stream of each service falls within a mutually independent range in which the hierarchy of pictures to be included in the basic stream and the extended stream is assigned in advance.

  FIG. 21 shows another example of the change in the hierarchical coding structure accompanying the service switching operation. In this example, layers 0 to 3 are assigned to the basic stream, and layers 5 and 6 are assigned to the extension stream.

  In this example, the video stream of the 60p service includes encoded image data of each picture subjected to hierarchical encoding. In this case, there are 0 to 3 picture layers. The 60p service video stream includes only the basic stream. This basic stream includes pictures of all layers from 0 to 3. In this case, “general_level_idc” is set to “level 5.1” and “sps_max_sublayer_minus1 = 3” in the SPS of the basic stream.

  In this example, the 120p service video stream includes encoded image data of each picture that has been hierarchically encoded. In this case, there are pictures of 0 to 2 and 5 layers. The 120p service video stream includes a basic stream and an extended stream. The basic stream includes pictures in layers 0 to 2, and the extended stream includes pictures in layer 5. In this case, “general_level_idc” is set to “level5.2” and “sps_max_sublayer_minus1 = 5” in the SPS (Sequence Parameter Set) of the basic stream.

  At the timing immediately before the 60p service transmission period starts, a PMT having information on the video stream of the 60p service is inserted into the container layer. This PMT includes basic stream and extended stream information. In the HEVC descriptor related to the basic stream, “level_idc” is set to “level 5.1”, and further, “temporal_id_min = 0” and “temporal_id_max = 3” are set.

  In the HEVC descriptor related to the extension stream, “level_idc” is set to “level 5.2”, and further, “temporal_id_min = 5” and “temporal_id_max = 6” are set. This setting indicates that the hierarchical range that can be taken by the extension stream is 5 or 6, and that no extension stream actually exists. Note that, as shown in FIG. 20, the basic stream information may not be included in this PMT.

  In the 60p service transmission period, a PMT having information on the video stream of the 120p service is inserted into the container layer at a timing immediately before the 120p service is started. In the HEVC descriptor related to the basic stream included in the PMT, “level_idc” is set to “level 5.1”, and further, “temporal_id_min = 0” and “temporal_id_max = 3” are set. In the HEVC descriptor related to the extension stream included in this PMT, “level_idc” is set to “level 5.2”, and further, “temporal_id_min = 5” and “temporal_id_max = 5” are set.

  In this case, in the period T in which the 60p service transmission period ends from the insertion of the PMT having the 120p service video stream information, the video layer information is the 60p service video stream information, whereas the system layer information The information becomes information of a 120p service video stream. However, in this example, the range of the “temporal_id” value of the video matches the range of the “temporal_id” value of the HEVC descriptor. This is because hierarchical coding is performed so that the video stream of each service falls within a mutually independent range in which the hierarchy of pictures to be included in the basic stream and the extended stream is assigned in advance.

  FIG. 22 shows an example of a change in the hierarchical coding structure accompanying the service switching operation. In this example, the layers of pictures included in the basic stream and the extension stream are not assigned in advance.

  In this example, the video stream of the 60p service includes encoded image data of each picture subjected to hierarchical encoding. In this case, there are 0 to 3 picture layers. The 60p service video stream includes only the basic stream. This basic stream includes pictures of all layers from 0 to 3. In this case, “general_level_idc” is set to “level 5.1” and “sps_max_sublayer_minus1 = 3” in the SPS of the basic stream.

  In this example, the 120p service video stream includes encoded image data of each picture that has been hierarchically encoded. In this case, there are 0 to 3 picture layers. The 120p service video stream includes a basic stream and an extended stream. The basic stream includes pictures in layers 0 to 2, and the extended stream includes pictures in layer 3. In this case, “general_level_idc” is set to “level5.2” and “sps_max_sublayer_minus1 = 3” in the SPS (Sequence Parameter Set) of the basic stream.

  At the timing immediately before the 60p service transmission period starts, a PMT having information on the video stream of the 60p service is inserted into the container layer. In the HEVC descriptor (HEVC descriptor) included in this PMT, “level_idc” is set to “level 5.1”, and further, “temporal_id_min = 0” and “temporal_id_max = 3” are set.

  Similarly, in the 60p service transmission period, a PMT having information on the video stream of the 120p service is inserted into the container layer at a timing immediately before the 120p service is started. In the HEVC descriptor related to the basic stream included in this PMT, “level_idc” is set to “level 5.1”, and “temporal_id_min = 0” and “temporal_id_max = 2” are set. In the HEVC descriptor related to the extension stream included in this PMT, “level_idc” is set to “level 5.2”, and further, “temporal_id_min = 3” and “temporal_id_max = 3” are set.

  In this case, in the period T in which the 60p service transmission period ends from the insertion of the PMT having the 120p service video stream information, the video layer information is the 60p service video stream information, whereas the system layer information The information becomes information of a 120p service video stream. In this example, the range of the “temporal_id” value of the video and the “temporal_id” value of the HEVC descriptor do not match. This is because the video stream of each service is not subjected to hierarchical coding so that the picture layers included in the basic stream and the extended stream are within the mutually independent range, and the layer with temporal_ID = 3 This is because it fluctuates in the basic stream or in the extension stream.

  As described above, in the transmission / reception system 10 shown in FIG. 1, the transmission apparatus 100 assigns a fixed identifier (PID) to the basic stream and the extension stream. As a result, the receiving side does not need to change the setting of the filter for extracting each stream by the demultiplexer 203 even when switching from the 60p service to 120p or from the 120p service to the 60p service. Occurrence of display mute can be suppressed, and seamless display becomes possible.

  Also, in the transmission / reception system 10 shown in FIG. 1, this is because the video streams of each service fall within the mutually independent ranges in which the layers of pictures included in the basic stream and the extension stream are assigned in advance. Stream encoding is performed. For this reason, it is possible to avoid a mismatch between the “temporal_id” value of the video and the “temporal_id” value range of the HEVC descriptor.

<2. Modification>
In the above-described embodiment, the transmission / reception system 10 including the transmission device 100 and the reception device 200 is shown, but the configuration of the transmission / reception system to which the present technology can be applied is not limited to this. For example, the receiving device 200 may have a configuration of a set top box and a monitor connected by a digital interface such as HDMI (High-Definition Multimedia Interface). “HDMI” is a registered trademark.

  Further, in the above-described embodiment, an example in which the container is a transport stream (MPEG-2 TS) is shown. However, the present technology can be similarly applied to a system configured to be distributed to receiving terminals using a network such as the Internet. In Internet distribution, MP4 or other format containers are often distributed. In other words, containers of various formats such as transport stream (MPEG-2 TS) adopted in the digital broadcasting standard and MP4 used in Internet distribution correspond to the container.

Moreover, this technique can also take the following structures.
(1) an image encoding unit that encodes image data of each picture constituting moving image data to generate a video stream;
A transmission unit that transmits a container of a predetermined format that continuously includes the first video stream and the second video stream generated by the image encoding unit;
The first video stream and the second video stream are composed of first to Mth (M ≦ N) streams among the first to Nth,
The transmitter is
A transmission apparatus that assigns a fixed identifier to each of the first to Nth streams.
(2) The transmission unit
In the transmission period of the first video stream,
Always, the identifier information of each stream constituting the first video stream is inserted into the container layer and transmitted.
The transmission device according to (1), wherein identification information of each stream constituting the second video stream is inserted into a container layer and transmitted at a timing immediately before the transmission period of the second video stream is started. .
(3) The transmission unit
In the transmission period of each video stream,
The transmission apparatus according to (1), wherein the identification information of each of the first to Nth streams is inserted into a container layer and transmitted.
(4) The transmission unit
In the transmission period of each video stream,
The transmission device according to any one of (1) to (3), wherein the switching information is inserted into a container layer and transmitted.
(5) The transmission device according to (4), wherein the switching information includes frame rate information and / or stream configuration information of encoded image data included in the second video stream.
(6) The transmission device according to any one of (1) to (5), wherein the first video stream and the second video stream are configured by at least a basic stream of a basic stream and an extended stream.
(7) A video stream composed of the basic stream has encoded image data of the first frame rate,
The transmission apparatus according to (6), wherein the video stream configured by the basic stream and the extension stream has encoded image data having a second frame rate that is twice the first frame rate.
(8) The image encoding unit
Classifying the image data of each picture constituting the moving image data into a plurality of layers, encoding the image data of the classified pictures of each layer, and dividing the plurality of layers into M layer sets, Generating the first to Mth streams of the video streams respectively having the encoded image data of the divided pictures of each layer set;
The transmission apparatus according to any one of (1) to (7), wherein hierarchical encoding is performed so that a hierarchy of pictures to be included in each of the first to Mth streams falls within a mutually assigned range that is assigned in advance. .
(9) The transmission unit
In the transmission period of the first video stream,
Always, the hierarchical range information of the pictures of each stream constituting the first video stream is inserted into the container layer and transmitted.
The layer range information of the pictures of each stream constituting the second video stream is inserted into the container layer and transmitted at a timing immediately before the transmission period of the second video stream is started. Transmitter.
(10) The first video stream and the second video stream include at least a basic stream of a basic stream and an extension stream,
The image encoding unit is
The transmission apparatus according to (8) or (9), wherein encoding is performed so that a picture layer included in the extension stream is one.
(11) The transmission unit
In the transmission period of each video stream,
When the video stream includes only the basic stream,
The transmission apparatus according to (10), wherein the hierarchical range information of the picture of the extended stream is inserted into the container layer and transmitted together with the hierarchical range information of the picture of the basic stream.
(12) an image encoding step of generating video streams by encoding image data of each picture constituting moving image data;
A transmission step of transmitting a container of a predetermined format that continuously includes the first video stream and the second video stream generated in the image encoding step;
The first video stream and the second video stream are composed of first to Mth (M ≦ N) streams among the first to Nth,
In the sending step above,
A transmission method for assigning a fixed identifier to each of the first to Nth streams.
(13) A receiving unit that receives a container of a predetermined format that continuously includes the first video stream and the second video stream having encoded image data,
The first video stream and the second video stream are composed of first to Mth (M ≦ N) streams among the first to Nth,
A fixed identifier is given to each of the first to Nth streams,
A receiving apparatus, further comprising: a processing unit configured to filter and process each stream included in the first video stream and the second video stream based on an assigned identifier.
(14) The first to Mth streams of the video streams are
The image data of each picture constituting the moving image data is classified into a plurality of layers, the image data of the classified pictures of each layer is encoded, and the plurality of layers are divided into M layer sets. , Each having the encoded image data of the picture of each divided layer set,
The reception apparatus according to (13), wherein the hierarchy of pictures to be included in each of the first to Mth streams is hierarchically encoded so as to fall within a mutually independent range assigned in advance.
(15) a receiving step of receiving a container of a predetermined format that continuously includes the first video stream and the second video stream having encoded image data;
The first video stream and the second video stream are composed of first to Mth (M ≦ N) streams among the first to Nth,
A fixed identifier is given to each of the first to Nth streams,
A receiving method further comprising a processing step of filtering and processing each stream included in the first video stream and the second video stream based on an assigned identifier.
(16) an image encoding unit that generates a video stream having encoded image data;
A transmission unit that transmits a container of a predetermined format that continuously includes the first video stream and the second video stream generated by the image encoding unit;
The first video stream and the second video stream are composed of first to Mth (M ≦ N) streams among the first to Nth,
The image encoding unit is
Classifying the image data of each picture constituting the moving image data into a plurality of layers, encoding the image data of the classified pictures of each layer, and dividing the plurality of layers into M layer sets, Generating the first to Mth streams of the video streams respectively having the encoded image data of the divided pictures of each layer set;
A transmission apparatus that performs hierarchical encoding so that a hierarchy of pictures to be included in each of the first to M-th streams falls within a mutually independent range assigned in advance.
(17) The transmission unit
In the transmission period of the first video stream,
Always, the hierarchical range information of the pictures of each stream included in the first video stream is inserted into the container layer and transmitted,
The layer range information of the pictures of each stream included in the second video stream is inserted into the container layer and transmitted at a timing immediately before the transmission period of the second video stream is started. (16) Transmitter.
(18) The first video stream and the second video stream include at least a basic stream of a basic stream and an extension stream,
The transmission apparatus according to (16) or (17), wherein the picture included in the extension stream has one layer.
(19) The transmission unit
In the transmission period of each video stream,
When the video stream includes only the basic stream,
The transmission device according to (18), wherein the hierarchical range information of the picture of the extended stream is inserted into the container layer and transmitted together with the hierarchical range information of the picture of the basic stream.

  The main feature of this technology is that by assigning fixed identifiers to the basic stream and extension stream, the receiver can change the filter settings for extracting each stream with the demultiplexer even when the service is switched. In other words, the occurrence of display mute is suppressed, and seamless display is enabled (see FIG. 16). In addition, the main feature of the present technology is that the video stream of each service is hierarchically encoded so that the layers of pictures to be included in the basic stream and the extended stream are within the mutually independent ranges assigned in advance. This is to avoid the mismatch of the “temporal_id” value of the HEVC descriptor (see FIG. 20).

DESCRIPTION OF SYMBOLS 10 ... Transmission / reception system 100 ... Transmission apparatus 101 ... CPU
102A, 102B... Encoder 103A, 103B... Compressed data buffer (cpb)
104 ... Multiplexer 105 ... Transmission unit 141 ... Section information generation unit 142 ... Null packet generation unit 143 ... Selector 144 ... PID allocation unit 145 ... TS multiplexing unit 200 ...・ Receiver 201 ... CPU
202: receiving unit 203 ... demultiplexer 204 ... compressed data buffer (cpb)
205: Decoder 206: Uncompressed data buffer (dpb)
207: Post processing unit 208: Display unit 231: PID filter 232_0 to 232_n, 232_null, 232_c: Multiplexing buffer 233: Section filter 234: PMT analysis unit

Claims (8)

Image code that hierarchically encodes the image data of each picture constituting the moving image data and generates an extended stream having encoded image data of the lower layer picture and encoded image data of the higher layer picture Equipped with
When the hierarchical structure of each picture of the basic stream changes, the image encoding unit sets a hierarchy of pictures included in the extended stream as a fixed hierarchy larger than a maximum hierarchy that the picture of the basic stream can take,
A multiplexed stream generating unit that generates a multiplexed stream including the basic stream and the extension stream generated by the image encoding unit;
An information insertion unit for inserting a bit rate level specification value of the basic stream into the multiplexed stream;
A transmission apparatus , further comprising: a transmission unit that transmits the multiplexed stream in which the bit stream level designation value of the basic stream is inserted . The information insertion part
Further inserting a bit rate level specification value that combines the basic stream and the extension stream into the multiplexed stream ,
The transmitter is
The transmission apparatus according to claim 1, wherein a multiplexed stream in which a bit rate level specification value obtained by combining the basic stream and the extension stream is inserted together with a bit rate level specification value of the basic stream is transmitted. The information insertion part
Further inserting the hierarchical range information of the pictures of the basic stream and the extended stream into the multiplexed stream ,
The transmitter is
2. The transmission apparatus according to claim 1, wherein a multiplexed stream in which hierarchical range information of pictures of the basic stream and the extended stream is inserted together with a level designation value of the bit rate of the basic stream is transmitted. The transmission device according to claim 1, wherein the basic stream has encoded image data of 60 Hz, and the basic stream and the extension stream have encoded image data of 120 Hz. The transmission apparatus according to claim 1, wherein the number of pictures included in the extension stream is one. The image encoding unit hierarchically encodes the image data of each picture constituting the moving image data, and has the basic stream having the encoded image data of the lower layer picture and the encoded image data of the higher layer picture. An image encoding step for generating a stream;
In the image encoding step, when the hierarchical structure of each picture of the basic stream changes, the image encoding unit determines a hierarchy of pictures included in the extension stream from a maximum hierarchy that the picture of the basic stream can take. A large fixed hierarchy,
A multiplexed stream generating unit for generating a multiplexed stream including the basic stream and the extension stream generated in the image encoding step;
An information insertion step in which an information insertion unit inserts a level designation value of the bit rate of the basic stream into the multiplexed stream;
Transmission unit, the transmission method further including a transmission steps the level specified value of the bit rate of the base stream is sent the inserted multiplexed stream. A basic stream having encoded image data of pictures on the lower layer side and an extended stream having encoded image data of pictures on the higher layer side, generated by hierarchically encoding the image data of each picture constituting the moving image data A receiving unit that receives a multiplexed stream including:
When the hierarchical structure of each picture of the basic stream changes, the hierarchy of pictures included in the extension stream is a fixed hierarchy larger than the maximum hierarchy that the picture of the basic stream can take,
The level designation value of the bit rate of the basic stream is inserted into the multiplexed stream,
Depending on the decoding capability, only the encoded image data of the pictures of each layer included in the basic stream is extracted, or the encoded image data of the pictures of each layer included in the basic stream and the above-mentioned included in the extension stream both extracted encoded image data of a picture of a fixed hierarchy provided in the al processing unit for executing a decoding process to the encoded image data of each picture issued extract,
The processing unit
As encoded image data of the picture of each layer included in the elementary stream, the receiving apparatus that to extract the encoded image data of the picture of the fixed hierarchy than the lower layer. The basic stream having the encoded image data of the lower layer picture and the encoded image data of the higher layer picture generated by the reception unit being hierarchically encoded with the image data of each picture constituting the moving image data Receiving a multiplexed stream including an extension stream having:
When the hierarchical structure of each picture of the basic stream changes, the hierarchy of pictures included in the extension stream is a fixed hierarchy larger than the maximum hierarchy that the picture of the basic stream can take,
The level designation value of the bit rate of the basic stream is inserted into the multiplexed stream,
The processing unit extracts only the encoded image data of the pictures of each layer included in the basic stream according to the decoding capability, or the encoded image data of the pictures of each layer included in the basic stream and the extension stream further have a process step of extracting both coded image data of the picture of the fixed hierarchy, it executes decoding processing on the encoded image data of each picture issued extract contained,
In the above processing steps,
A receiving method in which the processing unit extracts encoded image data of a picture in a lower layer than the fixed layer as encoded image data of a picture of each layer included in the basic stream .

Images