WO2017061070A1

WO2017061070A1 - Receiving device and receiving method

Info

Publication number: WO2017061070A1
Application number: PCT/JP2016/004020
Authority: WO
Inventors: 太峰中澤; 賀敬井口; 遠間　正真
Original assignee: パナソニックＩｐマネジメント株式会社
Priority date: 2015-10-07
Filing date: 2016-09-02
Publication date: 2017-04-13
Also published as: JP7209234B2; JP7429895B2; JP2024028530A; JP2022016499A; CN112954435A; JP2017073761A; CN112954435B; JP6986671B2; JP2023029431A

Abstract

A receiving device according to the present disclosure includes: a first processing unit that receives a broadcasting signal in which multiplexed data is modulated, wherein, among first multiplexed data in a first form of multiplexing and second multiplexed data in a second form of multiplexing, which differs from the first form of multiplexing, the multiplexed data is constituted of at least the first multiplexed data, that demodulates the received broadcasting signal, and that outputs the multiplexed data obtained through demodulation; and a conversion unit that converts, of the output multiplexed data, the form of multiplexing of the first multiplexed data into the second form of multiplexing and that outputs converted data obtained through conversion.

Description

Receiving apparatus and receiving method

The present disclosure relates to a receiving apparatus and a receiving method.

With the advancement of broadcasting and communication services, the introduction of ultra-high-definition video content such as 8K (7680 × 4320 pixels: hereinafter also referred to as 8K4K) and 4K (3840 × 2160 pixels: hereinafter also referred to as 4K2K) is considered. Has been. The receiving device needs to decode and display the encoded data of the received ultra-high-definition moving image in real time (real-time). Particularly, a moving image having a resolution of 8K or the like has a large processing load at the time of decoding, It is difficult to decode such a moving image in real time with one decoder. Therefore, a method of reducing real-time processing by reducing the processing load per decoder by parallelizing decoding processing using a plurality of decoders has been studied.

The encoded data is transmitted after being multiplexed based on a multiplexing method such as MPEG-2 TS (Transport Stream) or MMT (MPEG Media Transport). For example, Non-Patent Document 1 discloses a technique for transmitting encoded media data for each packet in accordance with MMT.

By the way, there are various multiplexing methods used for the TV broadcasting method, for example, there are a TS method and an IP multiplexing method as the multiplexing method. For this reason, in a receiving apparatus that supports a plurality of systems, when each system is individually mounted, the circuit scale is large and the cost increases.

Therefore, one aspect of the present disclosure provides a receiving device that can be easily shared with existing implementations and can be realized at low cost.

A receiving apparatus according to an aspect of the present disclosure includes at least the first multiplexing of first multiplexed data of a first multiplexing scheme and second multiplexed data of a second multiplexing scheme different from the first multiplexing scheme. Receiving a broadcast signal in which multiplexed data composed of data is modulated, demodulating the received broadcast signal, and outputting the multiplexed data obtained by demodulating, and output A conversion unit that converts the multiplexing method of the first multiplexed data out of the multiplexed data into the second multiplexing method and outputs the converted data obtained by the conversion.

A receiving device according to an aspect of the present disclosure is a second packet in which a first packet constituting first multiplexed data of a first multiplexing scheme is stored, and is different from the first multiplexing scheme A first conversion data composed of a second packet used in the second multiplexing method, and a second composed of a second packet obtained by converting the first packet into the second multiplexing method. A reception unit that receives conversion data multiplexed with conversion data, and a demultiplexing process that demultiplexes the conversion data received by the reception unit into the first conversion data and the second conversion data. And a demultiplexing unit that extracts the first packet from the second packet constituting the first converted data obtained by the demultiplexing process, and the extracted first packet constitutes the first packet For one data The first decoding unit that performs the first decoding process in the first multiplexing method, and the second data that is configured by the second packet that constitutes the second converted data obtained by the demultiplexing process On the other hand, a second decoding unit that performs a second decoding process in the second multiplexing scheme, a first decoding data obtained by the first decoding process, and a second decoding data obtained by the second decoding process And an output unit for outputting.

These general or specific aspects may be realized by a method, a system, an integrated circuit, a computer program, or a recording medium such as a computer-readable CD-ROM. The method, system, integrated circuit, computer program Also, any combination of recording media may be realized.

The receiving device of the present disclosure can be easily shared with existing implementations and can be realized at low cost.

FIG. 1 is a diagram illustrating an example of dividing a picture into slice segments. FIG. 2 is a diagram illustrating an example of a PES packet sequence in which picture data is stored. FIG. 3 is a diagram illustrating an example of picture division according to the first embodiment. FIG. 4 is a diagram illustrating an example of picture division according to the comparative example of the first embodiment. FIG. 5 is a diagram illustrating an example of data of the access unit according to the first embodiment. FIG. 6 is a block diagram of the transmission apparatus according to the first embodiment. FIG. 7 is a block diagram of the receiving apparatus according to the first embodiment. FIG. 8 is a diagram illustrating an example of the MMT packet according to the first embodiment. FIG. 9 is a diagram illustrating another example of the MMT packet according to the first embodiment. FIG. 10 is a diagram illustrating an example of data input to each decoding unit according to the first embodiment. FIG. 11 is a diagram illustrating an example of the MMT packet and header information according to the first embodiment. FIG. 12 is a diagram illustrating another example of data input to each decoding unit according to the first embodiment. FIG. 13 is a diagram illustrating an example of picture division according to the first embodiment. FIG. 14 is a diagram illustrating a flowchart of the transmission method according to the first embodiment. FIG. 15 is a block diagram of the receiving apparatus according to the first embodiment. FIG. 16 is a diagram illustrating a flowchart of the reception method according to the first embodiment. FIG. 17 is a diagram illustrating an example of the MMT packet and header information according to the first embodiment. FIG. 18 is a diagram illustrating an example of the MMT packet and header information according to the first embodiment. FIG. 19 is a diagram illustrating the configuration of the MPU. FIG. 20 is a diagram illustrating a configuration of MF metadata. FIG. 21 is a diagram for explaining the data transmission order. FIG. 22 is a diagram illustrating an example of a method for performing decoding without using header information. FIG. 23 is a block diagram of a transmission apparatus according to the second embodiment. FIG. 24 is a diagram illustrating a flowchart of the transmission method according to the second embodiment. FIG. 25 is a block diagram of a receiving apparatus according to the second embodiment. FIG. 26 is a flowchart of an operation for specifying the MPU head position and the NAL unit position. FIG. 27 is a diagram showing a flowchart of operations for acquiring initialization information based on the transmission order type and decoding media data based on the initialization information. FIG. 28 is a diagram illustrating a flowchart of the operation of the reception apparatus when the low delay presentation mode is provided. FIG. 29 is a diagram illustrating an example of the transmission order of MMT packets when auxiliary data is transmitted. FIG. 30 is a diagram for explaining an example in which the transmission apparatus generates auxiliary data based on the moof configuration. FIG. 31 is a diagram for explaining reception of auxiliary data. FIG. 32 is a diagram illustrating a flowchart of a reception operation using auxiliary data. FIG. 33 is a diagram showing a configuration of an MPU composed of a plurality of movie fragments. FIG. 34 is a diagram for explaining the transmission order of MMT packets when the MPU having the configuration of FIG. 33 is transmitted. FIG. 35 is a first diagram for describing an operation example of the receiving apparatus when one MPU is configured with a plurality of movie fragments. FIG. 36 is a second diagram for describing an operation example of the receiving device when one MPU is configured with a plurality of movie fragments. FIG. 37 is a flowchart of the operation of the reception method described with reference to FIGS. FIG. 38 is a diagram illustrating a case where non-VCL NAL units are individually set as data units and aggregated. FIG. 39 is a diagram illustrating a case where non-VCL NAL units are collectively used as a data unit. FIG. 40 is a diagram illustrating a flowchart of the operation of the reception device when packet loss occurs. FIG. 41 is a flowchart of the receiving operation when the MPU is divided into a plurality of movie fragments. FIG. 42 is a diagram illustrating an example of a picture prediction structure in each TemporalId when temporal scalability is realized. FIG. 43 is a diagram showing the relationship between the decoding time (DTS) and the display time (PTS) in each picture of FIG. FIG. 44 is a diagram illustrating an example of a picture prediction structure that requires picture delay processing and reorder processing. FIG. 45 is a diagram illustrating an example in which an MPU configured in the MP4 format is divided into a plurality of movie fragments and stored in an MMTP payload and an MMTP packet. FIG. 46 is a diagram for explaining a calculation method and problems of PTS and DTS. FIG. 47 is a diagram illustrating a flowchart of a reception operation when a DTS is calculated using information for DTS calculation. FIG. 48 is a diagram for explaining a method of storing data units in the payload in the MMT. FIG. 49 is a diagram illustrating an operation flow of the transmission apparatus according to the third embodiment. FIG. 50 is a diagram illustrating an operation flow of the reception device according to the third embodiment. FIG. 51 is a diagram illustrating an example of a specific configuration of the transmission apparatus according to the third embodiment. FIG. 52 is a diagram illustrating an example of a specific configuration of the reception apparatus according to the third embodiment. FIG. 53 is a diagram illustrating a method for storing non-timed media in the MPU and a method for transmitting the MMTP packet. FIG. 54 is a diagram illustrating an example in which a plurality of pieces of divided data obtained by dividing a file are packetized and transmitted. FIG. 55 is a diagram illustrating another example in which a plurality of pieces of divided data obtained by dividing a file are packetized and transmitted. FIG. 56 is a diagram showing the loop syntax for each file in the asset management table. FIG. 57 is a diagram showing an operation flow for specifying a divided data number in the receiving apparatus. FIG. 58 is a diagram showing an operation flow for specifying the number of divided data in the receiving apparatus. FIG. 59 is a diagram showing an operation flow for determining whether or not to operate the fragment counter in the transmission apparatus. FIG. 60 is a diagram for explaining a method for specifying the number of divided data and a divided data number (when a fragment counter is used). FIG. 61 is a diagram illustrating an operation flow of the transmission apparatus when the fragment counter is utilized. FIG. 62 is a diagram showing an operation flow of the receiving apparatus when the fragment counter is utilized. FIG. 63 is a diagram showing a service configuration when the same program is transmitted with a plurality of IP data flows. FIG. 64 is a diagram illustrating an example of a specific configuration of the transmission apparatus. FIG. 65 is a diagram illustrating an example of a specific configuration of the reception device. FIG. 66 is a diagram illustrating an operation flow of the transmission device. FIG. 67 is a diagram illustrating an operation flow of the reception device. FIG. 68 is a diagram showing a reception buffer model based on the reception buffer model defined in ARIB STD-B60, particularly when only the broadcast transmission path is used. FIG. 69 is a diagram illustrating an example in which a plurality of data units are aggregated and stored in one payload. FIG. 70 is a diagram illustrating an example in which a plurality of data units are aggregated and stored in one payload, and a video signal in the NAL size format is used as one data unit. FIG. 71 is a diagram showing the structure of the payload of an MMTP packet that does not indicate the data unit length. FIG. 72 is a diagram showing an example of storing information indicating a flag and the length of the size area in the extended area assigned to each packet. FIG. 73 is a diagram illustrating an operation flow of the reception device. FIG. 74 is a diagram illustrating an example of a specific configuration of the transmission apparatus. FIG. 75 is a diagram illustrating an example of a specific configuration of the reception device. FIG. 76 is a diagram illustrating an operation flow of the transmission apparatus. FIG. 77 is a diagram illustrating an operation flow of the reception device. FIG. 78 is a diagram showing an MMT / TLV protocol stack defined in ARIB STD-B60. FIG. 79 is a diagram illustrating a configuration of a TLV packet. FIG. 80 is a diagram illustrating an example of a block diagram of the reception device. FIG. 81 is a diagram for explaining the time stamp descriptor. FIG. 82 is a diagram for explaining leap second adjustment. FIG. 83 is a diagram illustrating a relationship among NTP time, MPU time stamp, and MPU presentation timing. FIG. 84 is a diagram for explaining a correction method for correcting the time stamp on the transmission side. FIG. 85 is a diagram for explaining a correction method for correcting a time stamp in the reception apparatus. FIG. 86 is a diagram showing an operation flow by the transmission side (transmission device) when the MPU time stamp is corrected on the transmission side (transmission device). FIG. 87 is a diagram showing an operation flow by the receiving device when the MPU time stamp is corrected on the transmitting side (transmitting device). FIG. 88 is a diagram showing an operation flow by the transmission side (transmission apparatus) when the MPU time stamp is corrected in the reception apparatus. FIG. 89 is a diagram illustrating an operation flow by the receiving apparatus when the MPU time stamp is corrected in the receiving apparatus. FIG. 90 is a diagram illustrating an example of a specific configuration of the transmission apparatus. FIG. 91 is a diagram illustrating an example of a specific configuration of the reception device. FIG. 92 is a diagram illustrating an operation flow of the transmission apparatus. FIG. 93 is a diagram illustrating an operation flow of the reception device. FIG. 94 is a diagram illustrating an extension example of the MPU extension time stamp descriptor. FIG. 95 is a diagram for describing a case where discontinuity occurs in the MPU sequence number due to the adjustment of the MPU sequence number. FIG. 96 is a diagram for describing a case where packet sequence numbers become discontinuous at the timing of switching from normal equipment to redundant equipment. FIG. 97 is a diagram showing an operation flow by the receiving apparatus when a discontinuity of the MPU sequence number or the packet sequence number occurs. FIG. 98 is a diagram for explaining a correction method for correcting a time stamp in the receiving apparatus when a leap second is inserted. FIG. 99 is a diagram for explaining a correction method for correcting a time stamp in the receiving apparatus when leap seconds are deleted. FIG. 100 is a diagram illustrating an operation flow of the reception apparatus. FIG. 101 is a diagram illustrating an example of a specific configuration of the transmission / reception system. FIG. 102 is a diagram illustrating a specific configuration of the reception apparatus. FIG. 103 is a diagram illustrating an operation flow of the reception apparatus. FIG. 104 is a diagram for explaining a correction method for correcting a time stamp on the transmission side (transmission apparatus) when a leap second is inserted. FIG. 105 is a diagram for explaining a correction method for correcting a time stamp on the transmission side (transmission apparatus) when leap seconds are deleted. FIG. 106 is a diagram showing an operation flow of the transmission apparatus described in FIG. FIG. 107 is a diagram showing an operation flow of the receiving apparatus described in FIG. FIG. 108 is a diagram showing an operation flow of the transmission apparatus described in FIG. FIG. 109 is a diagram illustrating an operation flow of the reception apparatus described in FIG. FIG. 110 is a diagram illustrating an example of a specific configuration of the transmission apparatus. FIG. 111 is a diagram illustrating an example of a specific configuration of the reception apparatus. FIG. 112 is a diagram illustrating an operation flow (transmission method) by the transmission apparatus. FIG. 113 is a diagram illustrating an operation flow (reception method) by the reception apparatus. FIG. 114 is a diagram showing details of a protocol stack diagram of the MMT / TLV system defined in ARIB STD-B60. FIG. 115 is a diagram illustrating a block diagram of the reception device. FIG. 116 is a diagram showing a general broadcast protocol multiplexed in MPEG-2 TS Systems. FIG. 117 is a block diagram of a receiving apparatus that receives broadcast signals multiplexed by the TS method. FIG. 118A is a diagram showing a configuration of a conventional receiving apparatus when processing a single method. FIG. 118B is a diagram illustrating a configuration of a conventional receiving apparatus that independently processes a plurality of schemes. FIG. 118C is a diagram illustrating an example of a configuration of a reception device in the case of using a method conversion unit. FIG. 118D is a diagram illustrating another example of a configuration of a reception device when using a scheme conversion unit. FIG. 118E is a diagram illustrating another example of a configuration of a reception device in the case where a scheme conversion unit is used. FIG. 118F is a diagram illustrating another example of the configuration of the reception device in the case where the method conversion unit is used. FIG. 119 is a diagram illustrating a modified example of the configuration of the receiving device when the method conversion unit is used. FIG. 120 is a diagram illustrating an example of details of the method conversion unit. FIG. 121 is a diagram illustrating a reception processing flow received using the reception device illustrated in FIG. 118F. FIG. 122 is a diagram showing a processing flow of the method conversion means in FIG. FIG. 123 is a diagram showing a processing flow of the method processing means B in FIG. FIG. 124 is a diagram illustrating an example of a specific configuration of the reception apparatus. FIG. 125 is a diagram illustrating another example of the specific configuration of the reception device. FIG. 126 is a diagram illustrating an operation flow (reception method) by the reception apparatus. FIG. 127 is a diagram illustrating an operation flow (reception method) by the reception device.

Such a receiving device can be easily shared with existing implementations and can be realized at low cost.

In addition, the conversion unit extracts a part of the first multiplexed data, and the first packet constituting the first data is a second packet used in the second multiplexing method. A first conversion unit that performs first conversion stored in the first conversion unit and outputs first conversion data composed of the second packet obtained by the first conversion; and a remaining part of the first multiplexed data The first packet constituting the second data is extracted, the second conversion is performed to convert the extracted first packet into the second packet of the second multiplexing method, and obtained by the second conversion A second conversion unit that outputs second conversion data constituted by the second packet; and a multiplexing unit that performs a multiplexing process for multiplexing the output first conversion data and the second conversion data. And obtained by the multiplexing process The chromatography data, may be output as the converted data.

Further, the first data and the second data may be data of different media.

The first converter assigns a first identifier indicating the second packet obtained by the first conversion to the second packet of the first conversion data, and the second converter A second identifier indicating the second packet obtained by the second conversion may be added to the second packet of the second conversion data.

Furthermore, a retransmitting unit that retransmits the converted data output by the converting unit to another receiving device may be provided.

Further, an accumulating unit that accumulates the converted data output by the converting unit in a storage device may be provided.

Further, a retransmitting unit that retransmits the converted data stored in the storage device to another receiving device may be provided.

Furthermore, a second processing unit that performs a decoding process for decoding the conversion data output by the conversion unit and outputs the decoded data obtained by the decoding process may be provided.

Further, the image processing apparatus further includes a second processing unit that performs a decoding process for decoding the conversion data output by the conversion unit and outputs the decoded data obtained by the decoding process, wherein the second processing unit includes the conversion data A demultiplexing unit that performs a demultiplexing process that demultiplexes the converted data output by the unit into the first converted data and the second converted data, and the first obtained by the demultiplexing process A first decoding unit that performs a first decoding process in the first multiplexing scheme on the first data that is configured by the first packet extracted from the second packet that constitutes one conversion data; Second decoding by the second multiplexing method with respect to the second data constituting the second packet constituting the second converted data obtained by the demultiplexing process A second decoding unit that performs processing, and outputs the first decoded data obtained by the first decoding process and the second decoded data obtained by the second decoding process as the decoded data. Also good.

In addition, the receiving apparatus further uses the first control information of the first decoded data and the second control information of the second decoded data, to one of the first control information and the second control information. You may provide the adjustment part which performs adjustment which unites the other.

The first control information may be first reference clock information, the second control information may be second reference clock information, and the adjustment unit may include the first reference clock information and the second reference clock information. The first decoded data and the second decoded data may be synchronized by adjusting the other one to the other.

Also, the first multiplexing method may be an MMT / TLV method, and the second multiplexing method may be a TS method.

A receiving apparatus according to an aspect of the present disclosure is a second packet in which a first packet constituting first multiplexed data of a first multiplexing scheme is stored, and is a second packet different from the first multiplexing scheme. 1st conversion data comprised by the 2nd packet used by a multiplexing system, and 2nd conversion data comprised by the 2nd packet obtained by converting the said 1st packet to the said 2nd multiplexing system A receiving unit that receives the converted data multiplexed with each other, and a demultiplexing process that demultiplexes the converted data received by the receiving unit into the first converted data and the second converted data A demultiplexing unit, and the first packet extracted from the second packet constituting the first converted data obtained by the demultiplexing process, and the extracted first packet constitutes the first data Against the above A first decoding unit that performs a first decoding process in a multiplexing scheme, and a second data that is formed by the second packet that constitutes the second converted data obtained by the demultiplexing process, A second decoding unit for performing a second decoding process in the second multiplexing method; a first decoding data obtained by the first decoding process; and a second decoding data obtained by the second decoding process. An output unit.

Note that these comprehensive or specific aspects may be realized by a recording medium recording medium such as a method, a system, an integrated circuit, a computer program, or a computer-readable CD-ROM. You may implement | achieve with arbitrary combinations of a computer program or a recording medium.

Hereinafter, embodiments will be specifically described with reference to the drawings.

It should be noted that each of the embodiments described below shows a comprehensive or specific example. Numerical values, shapes, materials, components, arrangement positions and connection forms of components, steps, order of steps, and the like shown in the following embodiments are merely examples, and are not intended to limit the present disclosure. In addition, among the constituent elements in the following embodiments, constituent elements that are not described in the independent claims indicating the highest concept are described as optional constituent elements.

(Knowledge that became the basis of this disclosure)
In recent years, the resolution of displays such as TVs, smartphones, and tablet terminals has been increased. In particular, for broadcasting in Japan, a service of 8K4K (with a resolution of 8K × 4K) is scheduled for 2020. In the case of an ultra-high resolution moving image such as 8K4K, since it is difficult to decode in real time with a single decoder, a technique of performing decoding processing in parallel using a plurality of decoders has been studied.

Since the encoded data is multiplexed and transmitted based on a multiplexing method such as MPEG-2 TS or MMT, the receiving apparatus needs to separate the encoded data of the moving image from the multiplexed data before decoding. . Hereinafter, the process of separating the encoded data from the multiplexed data is referred to as demultiplexing.

When parallelizing the decoding process, it is necessary to distribute the encoded data to be decoded to each decoder. When the encoded data is distributed, it is necessary to analyze the encoded data itself, and particularly for content such as 8K, since the bit rate is very high, the processing load for analysis is large. Therefore, the demultiplexing part becomes a bottleneck, and there is a problem that reproduction in real time cannot be performed.

By the way, H.264 standardized by MPEG and ITU. H.264 and H.H. In a moving picture coding scheme such as H.265, the transmission apparatus can divide a picture into a plurality of areas called slices or slice segments, and perform coding so that each divided area can be decoded independently. Thus, for example, H. In the case of H.265, the receiving apparatus that receives the broadcast can separate the data for each slice segment from the received data, and output the data of each slice segment to a separate decoder, thereby realizing parallel decoding processing. .

FIG. 1 is a diagram illustrating an example in which one picture is divided into four slice segments in HEVC. For example, the receiving apparatus includes four decoders, and each decoder decodes one of four slice segments.

In conventional broadcasting, the transmission apparatus stores one picture (access unit in the MPEG system standard) in one PES packet, and multiplexes the PES packet into a TS packet sequence. Therefore, the receiving apparatus separates the payload of the PES packet and then analyzes the data of the access unit stored in the payload, thereby separating each slice segment and decoding the data of each separated slice segment. It was necessary to output to.

However, the inventor has found that it is difficult to perform this processing in real time due to the large amount of processing when analyzing the data of the access unit and separating the slice segments.

FIG. 2 is a diagram illustrating an example in which picture data divided into slice segments is stored in the payload of a PES packet.

As shown in FIG. 2, for example, data of a plurality of slice segments (slice segments 1 to 4) are stored in the payload of one PES packet. Further, the PES packet is multiplexed into a TS packet sequence.

(Embodiment 1)
In the following, H. An example of using H.265 will be described. The present embodiment can also be applied when using other encoding methods such as H.264.

FIG. 3 is a diagram illustrating an example in which an access unit (picture) according to the present embodiment is divided into division units. The access unit is H.264. The function called tiles introduced by H.265 is divided into two equal parts in the horizontal and vertical directions and divided into a total of four tiles. In addition, the slice segment and the tile are associated with each other one to one.

The reason for dividing into two in the horizontal and vertical directions will be described. First, at the time of decoding, a line memory for storing data of one horizontal line is generally required. However, when the resolution becomes very high such as 8K4K, the size in the horizontal direction increases, so the size of the line memory increases. In the implementation of the receiving apparatus, it is desirable that the size of the line memory can be reduced. In order to reduce the size of the line memory, vertical division is required. A vertical data division requires a data structure called tiles. For these reasons, tiles are used.

On the other hand, since an image generally has a high correlation in the horizontal direction, encoding efficiency is improved when a wide range can be referred to in the horizontal direction. Therefore, it is desirable that the access unit is divided in the horizontal direction from the viewpoint of coding efficiency.

¡By dividing the access unit into two equal parts in the horizontal and vertical directions, it is possible to achieve both of these two characteristics and to consider both the mounting surface and the coding efficiency. When a single decoder can decode a 4K2K moving image in real time, an 8K4K image is divided into four equal parts, and each slice segment is divided into 4K2K. Can decode 8K4K images in real time.

Next, the reason why the tiles obtained by dividing the access unit in the horizontal and vertical directions and the slice segments are associated one-to-one will be described. H. In H.265, an access unit is composed of a plurality of units called NAL (Network Adaptation Layer) units.

The payload of the NAL unit includes an access unit delimiter indicating the start position of the access unit, SPS (Sequence Parameter Set) which is initialization information commonly used in sequence units, and an initial decoding time commonly used in pictures. Information, PPS (Picture Parameter Set), SEI (Supplemental Enhancement Information) that is not necessary for the decoding process itself, but is necessary for decoding result processing and display, and encoded data of slice segments Is stored. The header of the NAL unit includes type information for identifying data stored in the payload.

Here, the transmission apparatus multiplexes the encoded data into a multiplexed format such as MPEG-2 TS, MMT (MPEG Media Transport), MPEG DASH (Dynamic Adaptive Streaming HTTP), or RTP (Real-time Transport Protocol). The basic unit can be set as a NAL unit. In order to store one slice segment in one NAL unit, it is desirable to divide the access unit into slice segments when dividing the access unit into regions. For this reason, the transmission apparatus associates tiles with slice segments on a one-to-one basis.

Note that, as shown in FIG. 4, the transmission apparatus can collectively set tiles 1 to 4 to one slice segment. However, in this case, all tiles are stored in one NAL unit, and it is difficult for the receiving apparatus to separate the tiles in the multiplexing layer.

The slice segment includes an independent slice segment that can be decoded independently and a reference slice segment that refers to the independent slice segment. Here, a case where the independent slice segment is used will be described.

FIG. 5 is a diagram showing an example of data of the access unit divided so that the boundary between the tile and the slice segment matches as shown in FIG. The access unit data includes the NAL unit storing the access unit delimiter arranged at the head, the SPS, PPS, and SEI NAL units arranged thereafter, and tiles 1 to 4 arranged thereafter. Data of slice segments in which data is stored. The access unit data may not include some or all of the SPS, PPS, and SEI NAL units.

Next, the configuration of transmitting apparatus 100 according to the present embodiment will be described. FIG. 6 is a block diagram showing a configuration example of transmitting apparatus 100 according to the present embodiment. The transmission apparatus 100 includes an encoding unit 101, a multiplexing unit 102, a modulation unit 103, and a transmission unit 104.

The encoding unit 101 outputs an input image, for example, H.264. Encoded data is generated by encoding according to H.265. In addition, for example, as illustrated in FIG. 3, the encoding unit 101 divides the access unit into four slice segments (tiles), and encodes each slice segment.

The multiplexing unit 102 multiplexes the encoded data generated by the encoding unit 101. Modulation section 103 modulates data obtained by multiplexing. Transmitter 104 transmits the modulated data as a broadcast signal.

Next, the configuration of receiving apparatus 200 according to the present embodiment will be described. FIG. 7 is a block diagram showing a configuration example of receiving apparatus 200 according to the present embodiment. The receiving apparatus 200 includes a tuner 201, a demodulating unit 202, a demultiplexing unit 203, a plurality of decoding units 204A to 204D, and a display unit 205.

Tuner 201 receives a broadcast signal. The demodulator 202 demodulates the received broadcast signal. The demodulated data is input to the demultiplexer 203.

The demultiplexing unit 203 separates the demodulated data into division units, and outputs the data for each division unit to the decoding units 204A to 204D. Here, the division unit is a divided area obtained by dividing the access unit. 265 is a slice segment. Here, the 8K4K image is divided into four 4K2K images. Therefore, there are four decoding units 204A to 204D.

The plurality of decoding units 204A to 204D operate in synchronization with each other based on a predetermined reference clock. Each decoding unit decodes the encoded data in units of division according to the DTS (Decoding Time Stamp) of the access unit, and outputs the decoding result to the display unit 205.

The display unit 205 generates an 8K4K output image by integrating a plurality of decoding results output from the plurality of decoding units 204A to 204D. The display unit 205 displays the generated output image according to the PTS (Presentation Time Stamp) of the access unit acquired separately. When integrating the decoding results, the display unit 205 performs a filtering process such as a deblocking filter so that the boundary is visually inconspicuous in a boundary region of adjacent division units such as a tile boundary. Also good.

In the above description, the transmission apparatus 100 and the reception apparatus 200 that transmit or receive broadcasts have been described as examples. However, content may be transmitted and received via a communication network. When receiving apparatus 200 receives content via a communication network, receiving apparatus 200 separates multiplexed data from IP packets received by a network such as Ethernet.

In broadcasting, the transmission path delay from when a broadcast signal is transmitted until it reaches the receiving device 200 is constant. On the other hand, in a communication network such as the Internet, the transmission path delay until data transmitted from the server reaches the receiving device 200 is not constant due to the influence of congestion. Therefore, the receiving apparatus 200 often does not perform strict synchronized playback based on a reference clock such as PCR in broadcast MPEG-2 TS. Therefore, the receiving apparatus 200 may display the 8K4K output image on the display unit in accordance with the PTS without strictly synchronizing the decoding units.

Also, due to the congestion of the communication network, all the division unit decoding processes may not be completed at the time indicated by the access unit PTS. In this case, the receiving apparatus 200 skips the display of the access unit or delays the display until the decoding of at least four division units is completed and the generation of the 8K4K image is completed.

Note that content may be transmitted and received using both broadcasting and communication. In addition, this method can be applied when reproducing multiplexed data stored in a recording medium such as a hard disk or a memory.

Next, a method for multiplexing access units divided into slice segments when MMT is used as a multiplexing method will be described.

FIG. 8 is a diagram showing an example of packetizing HEVC access unit data into MMT. SPS, PPS, SEI, and the like are not necessarily included in the access unit, but here, a case where they exist is illustrated.

The NAL units arranged before the first slice segment in the access unit such as the access unit delimiter, SPS, PPS, and SEI are collectively stored in the MMT packet # 1. Subsequent slice segments are stored in separate MMT packets for each slice segment.

Note that, as shown in FIG. 9, the NAL unit arranged before the first slice segment in the access unit may be stored in the same MMT packet as the first slice segment.

In addition, when a NAL unit such as End-of-Sequence or End-of-Bitstream indicating the end of a sequence or stream is added after the last slice segment, these are included in the same MMT packet as the last slice segment. Stored. However, since the NAL unit such as End-of-Sequence or End-of-Bitstream is inserted at the end point of the decoding process or the connection point of two streams, the receiving apparatus 200 inserts these NAL units. It may be desirable to be able to obtain easily in the multiplexing layer. In this case, these NAL units may be stored in an MMT packet different from the slice segment. Thereby, the receiving apparatus 200 can easily separate these NAL units in the multiplexing layer.

Note that TS, DASH, RTP, or the like may be used as a multiplexing method. Also in these schemes, the transmission device 100 stores different slice segments in different packets. This can ensure that the receiving apparatus 200 can separate slice segments in the multiplexing layer.

For example, when TS is used, encoded data is PES packetized in units of slice segments. When RTP is used, encoded data is RTP packetized in units of slice segments. Also in these cases, the NAL unit and the slice segment arranged before the slice segment may be packetized separately as in the MMT packet # 1 shown in FIG.

When TS is used, the transmission apparatus 100 indicates a unit of data stored in the PES packet by using a data alignment descriptor. Further, since DASH is a method of downloading data units in MP4 format called segments by HTTP or the like, the transmission apparatus 100 does not packetize encoded data in transmission. Therefore, transmitting apparatus 100 creates subsamples in units of slice segments so that receiving apparatus 200 can detect slice segments in the multiplexing layer in MP4, and stores information indicating the subsample storage position in the header of MP4. May be.

Hereinafter, the MMT packetization of slice segments will be described in detail.

As shown in FIG. 8, when the encoded data is packetized, data that is commonly referred to when decoding all slice segments in the access unit such as SPS and PPS is stored in the MMT packet # 1. In this case, receiving apparatus 200 concatenates the payload data of MMT packet # 1 and the data of each slice segment, and outputs the obtained data to the decoding unit. Thus, the receiving apparatus 200 can easily generate input data to the decoding unit by concatenating the payloads of a plurality of MMT packets.

FIG. 10 is a diagram showing an example in which input data to the decoding units 204A to 204D is generated from the MMT packet shown in FIG. The demultiplexing unit 203 concatenates the payload data of the MMT packet # 1 and the MMT packet # 2, thereby generating data necessary for the decoding unit 204A to decode the slice segment 1. The demultiplexing unit 203 similarly generates input data for the decoding unit 204B to the decoding unit 204D. That is, the demultiplexing unit 203 generates the input data of the decoding unit 204B by concatenating the payload data of the MMT packet # 1 and the MMT packet # 3. The demultiplexing unit 203 generates input data for the decoding unit 204C by concatenating the payload data of the MMT packet # 1 and the MMT packet # 4. The demultiplexing unit 203 generates the input data of the decoding unit 204D by concatenating the payload data of the MMT packet # 1 and the MMT packet # 5.

The demultiplexing unit 203 removes NAL units that are not necessary for the decoding process, such as access unit delimiters and SEI, from the payload data of the MMT packet # 1, and only the NPS units of SPS and PPS that are necessary for the decoding process. May be separated and added to the slice segment data.

When the encoded data is packetized as shown in FIG. 9, demultiplexing section 203 outputs MMT packet # 1 including the head data of the access unit to the first decoding section 204A in the multiplexing layer. . Further, the demultiplexing unit 203 analyzes the MMT packet including the head data of the access unit in the multiplexing layer, separates the SPS and PPS NAL units, and separates the separated SPS and PPS NAL units into the second and subsequent slices. By adding to each segment data, input data for each of the second and subsequent decoding units is generated.

Further, the receiving apparatus 200 uses the information included in the header of the MMT packet, the type of data stored in the MMT payload, and the slice segment in the access unit when the slice segment is stored in the payload. It is desirable to be able to identify the index number. Here, the data type refers to the data before the slice segment (NAL units arranged in the access unit before the first slice segment are collectively referred to in this way), and the data of the slice segment Either way. When a unit obtained by fragmenting an MPU such as a slice segment is stored in an MMT packet, a mode for storing an MFU (Media Fragment Unit) is used. When this mode is used, the transmission apparatus 100, for example, converts a Data Unit, which is a basic unit of data in the MFU, to a sample (a data unit in the MMT and corresponds to an access unit) or a subsample (sample (Divided units).

At this time, the header of the MMT packet includes a field called Fragmentation indicator and a field called Fragment counter.

The Fragmentation indicator indicates whether the data stored in the payload of the MMT packet is a fragment of the data unit, and if the fragment is a fragment, the fragment is the first or last fragment in the data unit, or Indicates whether the fragment is neither the beginning nor the end. In other words, the fragmentation indicator included in the header of a packet is (1) only the packet is included in the data unit that is the basic data unit, (2) the data unit is divided into a plurality of packets, and stored. The packet is the first packet of the Data unit, (3) the Data unit is divided into a plurality of packets and stored, and the packet is a packet other than the first and last of the Data unit, and (4) This is identification information indicating whether the Data unit is divided into a plurality of packets and stored, and the packet is the last packet of the Data unit.

“Fragment counter” is an index number indicating what number of fragment the data stored in the MMT packet corresponds to in the Data unit.

Therefore, the transmitting apparatus 100 sets the MMT sample in the Data unit, and sets the data before the slice segment and each slice segment in the Data unit fragment unit, so that the receiving apparatus 200 sets the header of the MMT packet. Can be used to identify the type of data stored in the payload. That is, the demultiplexing unit 203 can generate input data to the decoding units 204A to 204D with reference to the header of the MMT packet.

FIG. 11 is a diagram illustrating an example in which a sample is set to a Data unit, and data before a slice segment and a slice segment are packetized as a Data unit fragment.

The data before the slice segment and the slice segment are divided into five fragments from fragment # 1 to fragment # 5. Each fragment is stored in a separate MMT packet. At this time, the values of the Fragmentation indicator and the Fragment counter included in the header of the MMT packet are as illustrated.

For example, the Fragment indicator is a binary 2-bit value. The Fragment indicator of the MMT packet # 1 that is the head of the Data unit, the Fragment indicator of the final MMT packet # 5, and the Fragment indicator from the MMT packet # 2 to the MMT packet # 4 that are in between are different from each other. Set to a value. Specifically, the Fragment indicator of the MMT packet # 1 that is the head of the Data unit is set to 01, and the Fragment indicator of the final MMT packet # 5 is set to 11, and the MMT packet # 2 that is a packet in between The Fragment indicator up to MMT packet # 4 is set to 10. When only one MMT packet is included in the Data unit, the Fragment indicator is set to 00.

Fragment counter is 4 which is a value obtained by subtracting 1 from 5 which is the total number of fragments in MMT packet # 1, decreases by 1 in order in subsequent packets, and 0 in last MMT packet # 5. is there.

Therefore, the receiving apparatus 200 can identify the MMT packet storing the pre-slice segment data using either the Fragment indicator or the Fragment counter. Also, the receiving apparatus 200 can identify the MMT packet storing the Nth slice segment by referring to the Fragment counter.

The header of the MMT packet separately includes the sequence number in the MPU of the Movie Fragment to which the Data unit belongs, the sequence number of the MPU itself, and the sequence number in the Movie Fragment of the sample to which the Data unit belongs. The demultiplexing unit 203 can uniquely determine the sample to which the Data unit belongs by referring to these.

Furthermore, since the demultiplexing unit 203 can determine the index number of the fragment in the Data unit from the Fragment counter or the like, the slice segment stored in the fragment can be uniquely specified even when a packet loss occurs. For example, the demultiplexing unit 203 knows that the fragment received after the fragment # 3 is the fragment # 5 even when the fragment # 4 shown in FIG. Can be correctly output not to the decoding unit 204C but to the decoding unit 204D.

When a transmission path that guarantees no packet loss is used, the demultiplexing unit 203 refers to the header of the MMT packet, and the type of data stored in the MMT packet or the slice segment What is necessary is to process the arriving packet periodically without determining the index number. For example, when the access unit is transmitted by pre-slice data and a total of five MMT packets of four slice segments, the reception apparatus 200 determines the pre-slice data of the access unit to start decoding. By processing the received MMT packets in order, the pre-slice data and the data of the four slice segments can be acquired in order.

Hereinafter, a modification example of packetization will be described.

The slice segment does not necessarily have to be divided in both the horizontal direction and the vertical direction in the plane of the access unit, and may be obtained by dividing the access unit only in the horizontal direction as shown in FIG. However, it may be divided only in the vertical direction.

Also, if the access unit is divided only in the horizontal direction, tiles need not be used.

Also, the number of in-plane divisions in the access unit is arbitrary and is not limited to four. However, the slice segment and tile area sizes are H.264. It must be equal to or higher than the lower limit of the encoding standard such as H.265.

The transmission apparatus 100 may store identification information indicating an in-plane division method in the access unit in an MMT message, a TS descriptor, or the like. For example, information indicating the number of divisions in the horizontal direction and the vertical direction in the plane may be stored. Or, identification information unique to the division method, such as being divided into two equal parts in the horizontal direction and the vertical direction as shown in FIG. 3, or being divided into four equal parts in the horizontal direction as shown in FIG. May be assigned. For example, when the access unit is divided as shown in FIG. 3, the identification information indicates mode 1, and when the access unit is divided as shown in FIG. 1, the identification information indicates mode 1. .

Also, information indicating the restriction of the encoding condition related to the in-plane division method may be included in the multiplexing layer. For example, information indicating that one slice segment is composed of one tile may be used. Alternatively, the reference block when motion compensation is performed at the time of decoding a slice segment or tile is limited to the slice segment or tile at the same position in the screen, or limited to a block within a predetermined range in the adjacent slice segment. Information indicating this may be used.

Further, the transmission apparatus 100 may switch whether to divide the access unit into a plurality of slice segments according to the resolution of the moving image. For example, the transmitting apparatus 100 does not perform in-plane division when the processing target moving image has a resolution of 4K2K, and may divide the access unit into four when the processing target moving image is 8K4K. Good. By predefining the division method in the case of an 8K4K moving image, the receiving apparatus 200 determines the presence / absence of in-plane division and the division method by acquiring the resolution of the received moving image, and performs decoding. The operation can be switched.

Also, the receiving apparatus 200 can detect the presence or absence of in-plane division by referring to the header of the MMT packet. For example, when the access unit is not divided, if the MMT Data unit is set as a sample, the Data unit fragment is not performed. Therefore, the receiving apparatus 200 can determine that the access unit is not divided when the value of the Fragment counter included in the header of the MMT packet is always zero. Alternatively, the receiving apparatus 200 may detect whether the value of the fragmentation indicator is always 01. The receiving apparatus 200 can determine that the access unit is not divided even when the value of the fragmentation indicator is always 01.

Also, the receiving apparatus 200 can cope with a case where the number of in-plane divisions in the access unit does not match the number of decoding units. For example, when the receiving apparatus 200 includes two

decoding units

204A and 204B that can decode 8K2K encoded data in real time, the demultiplexing unit 203 transmits 8K4K encoded data to the decoding unit 204A. Two of the four slice segments that constitute

FIG. 12 is a diagram illustrating an operation example in the case where the data converted into the MMT packet as illustrated in FIG. 8 is input to the two

decoding units

204A and 204B. Here, it is desirable that the receiving apparatus 200 can output the decoding results of the

decoding units

204A and 204B as they are. Therefore, the demultiplexing unit 203 selects slice segments to be output to the

decoding units

204A and 204B so that the decoding results of the

decoding units

204A and 204B are spatially continuous.

Also, the demultiplexing unit 203 may select a decoding unit to be used according to the resolution or frame rate of the encoded data of the moving image. For example, when the receiving device 200 includes four 4K2K decoding units, if the resolution of the input image is 8K4K, the receiving device 200 performs a decoding process using all four decoding units. In addition, when the resolution of the input image is 4K2K, the receiving apparatus 200 performs the decoding process using only one decoding unit. Alternatively, the demultiplexing unit 203 integrates all the division units into a single decoding unit when 8K4K can be decoded in real time by a single decoding unit even if the in-plane is divided into four. Output to.

Furthermore, the receiving apparatus 200 may determine a decoding unit to be used in consideration of the frame rate. For example, when the receiving apparatus 200 includes two decoding units whose upper limit of the frame rate that can be decoded in real time is 60 fps when the resolution is 8K4K, encoded data of 120 fps at 8K4K may be input. is there. At this time, if the in-plane is composed of four division units, slice segment 1 and slice segment 2 are input to decoding section 204A, and slice segment 3 and slice segment 4 are The data is input to the decoding unit 204B. Each of the

decoding units

204A and 204B can decode up to 120 fps in real time if the resolution is 8K2K (the resolution is half of 8K4K), and thus the decoding processing is performed by these two

decoding units

204A and 204B.

Also, even if the resolution and frame rate are the same, the profile or level in the encoding method, or H.264 H.264 or H.264 When the encoding method itself such as H.265 is different, the processing amount is different. Therefore, the receiving apparatus 200 may select a decoding unit to be used based on these pieces of information. Note that the receiving device 200 cannot decode all the encoded data received by broadcasting or communication, or when all the slice segments or tiles constituting the area selected by the user cannot be decoded, the decoding unit Slice segments or tiles that can be decoded within the processing range may be automatically determined. Alternatively, the receiving apparatus 200 may provide a user interface for the user to select an area to be decoded. At this time, the receiving apparatus 200 may display a warning message indicating that the entire area cannot be decoded, or may display information indicating the number of areas, slice segments, or tiles that can be decoded.

The above method can also be applied to a case where an MMT packet storing a slice segment of the same encoded data is transmitted and received using a plurality of transmission paths such as broadcasting and communication.

Also, the transmission apparatus 100 may perform encoding so that the areas of the slice segments overlap in order to make the division unit boundaries inconspicuous. In the example shown in FIG. 13, an 8K4K picture is divided into four slice segments 1 to 4. Each of the slice segments 1 to 3 is, for example, 8K × 1.1K, and the slice segment 4 is 8K × 1K. Adjacent slice segments overlap each other. By doing so, motion compensation at the time of encoding can be performed efficiently at the boundary in the case of four divisions indicated by dotted lines, so that the image quality of the boundary portion is improved. In this way, image quality degradation at the boundary is reduced.

In this case, the display unit 205 cuts out an 8K × 1K area from the 8K × 1.1K area and integrates the obtained areas. The transmission apparatus 100 separately transmits information indicating whether slice segments are overlapped and encoded and information indicating the overlap range in the multiplexed layer or encoded data. Also good.

Note that the same method can be applied when tiles are used.

Hereinafter, the operation flow of the transmission apparatus 100 will be described. FIG. 14 is a flowchart illustrating an operation example of the transmission device 100.

First, the encoding unit 101 divides a picture (access unit) into a plurality of slice segments (tiles) that are a plurality of regions (S101). Next, the encoding unit 101 generates encoded data corresponding to each of the plurality of slice segments by performing encoding so that each of the plurality of slice segments can be independently decoded (S102). Note that the encoding unit 101 may encode a plurality of slice segments with a single encoding unit, or may perform parallel processing with a plurality of encoding units.

Next, the multiplexing unit 102 multiplexes the plurality of pieces of encoded data by storing the plurality of pieces of encoded data generated by the encoding unit 101 in a plurality of MMT packets (S103). Specifically, as illustrated in FIG. 8 and FIG. 9, the multiplexing unit 102 converts a plurality of pieces of encoded data into a plurality of encoded data so that encoded data corresponding to different slice segments is not stored in one MMT packet. Store in MMT packet. In addition, as shown in FIG. 8, multiplexing section 102 uses control information used in common for all decoding units in a picture as a plurality of MMT packets # 2 to ## in which a plurality of encoded data is stored. 5 is stored in MMT packet # 1 different from 5. Here, the control information includes at least one of an access unit delimiter, SPS, PPS, and SEI.

Note that the multiplexing unit 102 may store the control information in the same MMT packet as any one of a plurality of MMT packets in which a plurality of encoded data is stored. For example, as illustrated in FIG. 9, the multiplexing unit 102 stores the control information in the first MMT packet (MMT packet # 1 in FIG. 9) among the plurality of MMT packets in which the plurality of encoded data is stored. May be.

Finally, the transmission device 100 transmits a plurality of MMT packets. Specifically, the modulation unit 103 modulates the data obtained by multiplexing, and the transmission unit 104 transmits the modulated data (S104).

FIG. 15 is a block diagram illustrating a configuration example of the receiving apparatus 200, and illustrates a detailed configuration of the demultiplexing unit 203 illustrated in FIG. 7 and its subsequent stage. As illustrated in FIG. 15, the reception device 200 further includes a decryption command unit 206. In addition, the demultiplexing unit 203 includes a type determination unit 211, a control information acquisition unit 212, a slice information acquisition unit 213, and a decoded data generation unit 214.

Hereinafter, the operation flow of the receiving apparatus 200 will be described. FIG. 16 is a flowchart illustrating an operation example of the reception device 200. Here, the operation for one access unit is shown. When the decoding process of a plurality of access units is executed, the process of this flowchart is repeated.

First, the receiving apparatus 200 receives, for example, a plurality of packets (MMT packets) generated by the transmitting apparatus 100 (S201).

Next, the type determination unit 211 acquires the type of encoded data stored in the received packet by analyzing the header of the received packet (S202).

Next, the type determination unit 211 determines whether the data stored in the received packet is pre-slice segment data or slice segment data based on the type of encoded data acquired (S203). .

When the data stored in the received packet is the data before the slice segment (Yes in S203), the control information acquisition unit 212 acquires the data before the slice segment of the access unit to be processed from the payload of the received packet, and the slice The pre-segment data is stored in the memory (S204).

On the other hand, when the data stored in the received packet is slice segment data (No in S203), the receiving apparatus 200 uses the header information of the received packet to store a plurality of data stored in the received packet. It is determined which area of the encoded data is the encoded data. Specifically, the slice information acquisition unit 213 acquires the index number Idx of the slice segment stored in the received packet by analyzing the header of the received packet (S205). Specifically, the index number Idx is an index number in the Movie Fragment of the access unit (sample in MMT).

Note that the processing in step S205 may be performed collectively in step S202.

Next, the decoded data generation unit 214 determines a decoding unit that decodes the slice segment (S206). Specifically, the index number Idx and a plurality of decoding units are associated in advance, and the decoded data generation unit 214 decodes the slice segment by using the decoding unit corresponding to the index number Idx acquired in step S205. The decoding unit to be determined is determined.

Note that, as described in the example of FIG. 12, the decoded data generation unit 214 includes the resolution of the access unit (picture), the method of dividing the access unit into a plurality of slice segments (tiles), and a plurality of reception devices 200. The decoding unit that decodes the slice segment may be determined based on at least one of the processing capabilities of the decoding unit. For example, the decoded data generation unit 214 determines an access unit dividing method based on identification information in an MMT message or a descriptor such as a TS section.

Next, the decoded data generation unit 214 includes control information used in common for all the decoding units in the picture included in any of the plurality of packets, and each of the plurality of pieces of encoded data of the plurality of slice segments. Are combined to generate a plurality of input data (combined data) to be input to a plurality of decoding units. Specifically, the decoded data generation unit 214 acquires slice segment data from the payload of the received packet. The decoded data generation unit 214 generates input data to the decoding unit determined in step S206 by combining the pre-slice segment data stored in the memory in step S204 and the acquired slice segment data. (S207).

After step S204 or S207, when the data of the received packet is not the final data of the access unit (No in S208), the processing after step S201 is performed again. That is, the above process is repeated until input data to a plurality of decoding units 204A to 204D corresponding to all slice segments included in the access unit is generated.

Note that the timing at which a packet is received is not limited to the timing shown in FIG. 16, and a plurality of packets may be received in advance or stored in a memory or the like.

On the other hand, when the data of the received packet is the final data of the access unit (Yes in S208), the decoding command unit 206 outputs the plurality of input data generated in Step S207 to the corresponding decoding units 204A to 204D. (S209).

Next, the plurality of decoding units 204A to 204D generate a plurality of decoded images by decoding a plurality of input data in parallel according to the DTS of the access unit (S210).

Finally, the display unit 205 generates a display image by combining a plurality of decoded images generated by the plurality of decoding units 204A to 204D, and displays the display image according to the PTS of the access unit (S211).

The receiving apparatus 200 acquires the DTS and PTS of the access unit by analyzing the payload data of the MMT packet storing the MPU header information or the movie fragment header information. In addition, when TS is used as the multiplexing method, the receiving apparatus 200 acquires the DTS and PTS of the access unit from the header of the PES packet. When RTP is used as a multiplexing method, the receiving apparatus 200 acquires the DTS and PTS of the access unit from the header of the RTP packet.

The display unit 205 may perform a filtering process such as a deblocking filter at the boundary between adjacent division units when integrating the decoding results of a plurality of decoding units. In addition, since the filtering process is not necessary when displaying the decoding result of the single decoding unit, the display unit 205 performs the process according to whether the filtering process is performed on the boundary of the decoding results of the plurality of decoding units. It may be switched. Whether the filtering process is necessary may be defined in advance according to the presence or absence of division. Or the information which shows whether a filter process is required may be separately stored in a multiplexing layer. In addition, information necessary for filter processing such as filter coefficients may be stored in SPS, PPS, SEI, or a slice segment. The decoding units 204A to 204D or the demultiplexing unit 203 acquires these information by analyzing the SEI, and outputs the acquired information to the display unit 205. The display unit 205 performs filter processing using these pieces of information. When these pieces of information are stored in the slice segment, it is desirable that the decoding units 204A to 204D obtain these pieces of information.

In the above description, an example in which there are two types of data stored in a fragment, that is, data before a slice segment and a slice segment, may be three or more types of data. In this case, case classification according to the type is performed in step S203.

Further, when the data size of the slice segment is large, the transmission apparatus 100 may fragment the slice segment and store it in the MMT packet. That is, the transmission apparatus 100 may fragment the pre-slice segment data and the slice segment. In this case, if the access unit and the Data unit are set equal as in the packetization example shown in FIG. 11, the following problem occurs.

For example, when the slice segment 1 is divided into three fragments, the slice segment 1 is divided into three packets having a fragment counter value of 1 to 3 and transmitted. In slice segment 2 and later, the Fragment counter value is 4 or more, and the Fragment counter value cannot be associated with the data stored in the payload. Therefore, the receiving apparatus 200 cannot identify the packet storing the head data of the slice segment from the information of the header of the MMT packet.

In such a case, the receiving apparatus 200 may identify the start position of the slice segment by analyzing the payload data of the MMT packet. Here, H. H.264 or H.264 In 265, as a format for storing the NAL unit in the multiplexing layer, a format called a byte stream format in which a start code consisting of a specific bit string is added immediately before the NAL unit header, and a field indicating the size of the NAL unit are added. There are two types, called NAL size format.

Byte stream format is used in MPEG-2 system and RTP. The NAL size format is used in MP4 and DASH and MMT that use MP4.

When the byte stream format is used, the receiving apparatus 200 analyzes whether or not the top data of the packet matches the start code. If the start data of the packet matches the start code, the receiving apparatus 200 acquires the type of the NAL unit from the subsequent NAL unit header, so that the data included in the packet is slice segment data. Whether it can be detected.

On the other hand, in the case of the NAL size format, the receiving apparatus 200 cannot detect the start position of the NAL unit based on the bit string. Therefore, in order to acquire the start position of the NAL unit, the receiving apparatus 200 needs to shift the pointer by reading data by the size of the NAL unit in order from the head NAL unit of the access unit.

However, in the MPU or Movie Fragment header in the MMT, the size of the subsample unit is indicated, and when the subsample corresponds to the pre-slice data or the slice segment, the receiving apparatus 200 determines based on the subsample size information. The starting position of each NAL unit can be specified. Therefore, the transmission apparatus 100 may include information indicating whether sub-sample information exists in the MPU or the Movie Fragment in information acquired by the reception apparatus 200 at the start of data reception, such as MPT in MMT. .

The MPU data is expanded based on the MP4 format. In MP4, H.P. H.264 or H.264 There are modes in which parameter sets such as 265 SPS and PPS can be stored as sample data, and modes in which they cannot be stored. Also, information for specifying this mode is shown as an entry name of SampleEntry. When the storable mode is used and the parameter set is included in the sample, the receiving apparatus 200 acquires the parameter set by the method described above.

On the other hand, when a mode that cannot be stored is used, the parameter set is stored as a Decoder Specific Information in SampleEntry, or is stored using a stream for the parameter set. Here, since the parameter set stream is not generally used, it is desirable that the transmission apparatus 100 stores the parameter set in the Decoder Specific Information. In this case, the receiving apparatus 200 analyzes the SampleEntry transmitted as MPU metadata or Movie Fragment metadata in the MMT packet, and acquires a parameter set referred to by the access unit.

When the parameter set is stored as sample data, the receiving apparatus 200 can acquire the parameter set necessary for decoding by referring to only the sample data without referring to the SampleEntry. At this time, the transmission apparatus 100 does not have to store the parameter set in the SampleEntry. By doing so, the transmission apparatus 100 can use the same SampleEntry in different MPUs, and therefore the processing load on the transmission apparatus 100 when generating an MPU can be reduced. Furthermore, there is a merit that the receiving apparatus 200 does not need to refer to the parameter set in the SampleEntry.

Alternatively, the transmitting apparatus 100 may store one default parameter set in SampleEntry and store the parameter set referred to by the access unit in the sample data. In conventional MP4, since a parameter set is generally stored in SampleEntry, if there is no parameter set in SampleEntry, there may be a receiving device that stops reproduction. This problem can be solved by using the above method.

Alternatively, the transmission apparatus 100 may store the parameter set in the sample data only when a parameter set different from the default parameter set is used.

In both modes, since the parameter set can be stored in SampleEntry, the transmitting apparatus 100 may always store the parameter set in VisualSampleEntry, and the receiving apparatus 200 may always acquire the parameter set from VisualSampleEntry.

In the MMT standard, MP4 header information such as Moov and Moof is transmitted as MPU metadata or movie fragment metadata. However, the transmission apparatus 100 does not necessarily transmit MPU metadata and movie fragment metadata. You don't have to. Further, the receiving apparatus 200 determines whether SPS and PPS are stored in the sample data based on ARIB (Association of Radio Industries and Businesses) standard service, asset type, or MPU meta transmission status. It is also possible to determine.

FIG. 17 is a diagram illustrating an example in which the data before the slice segment and each slice segment are set to different Data units.

In the example shown in FIG. 17, the data before the slice segment and the data sizes from the slice segment 1 to the slice segment 4 are Length # 1 to Length # 5, respectively. The field values of Fragmentation indicator, Fragment counter, and Offset included in the header of the MMT packet are as shown in the figure.

Here, Offset indicates the bit length (offset) from the beginning of the encoded data of the sample (access unit or picture) to which the payload data belongs to the first byte of the payload data (encoded data) included in the MMT packet. Offset information. The value of Fragment counter is described as starting from a value obtained by subtracting 1 from the total number of fragments, but may be started from other values.

FIG. 18 is a diagram illustrating an example in which the Data unit is fragmented. In the example shown in FIG. 18, slice segment 1 is divided into three fragments and stored in MMT packet # 2 to MMT packet # 4, respectively. Also at this time, assuming that the data size of each fragment is Length # 2_1 to Length # 2_3, the values of the fields are as shown in the figure.

Thus, when the data unit such as the slice segment is set to Data unit, the head of the access unit and the head of the slice segment can be determined as follows based on the field value of the MMT packet header.

The head of the payload in the packet whose Offset value is 0 is the head of the access unit.

The head of the payload of a packet whose Offset value is different from 0 and whose Fragmentation indicator value is 00 or 01 is the head of the slice segment.

In addition, when no Data unit fragmentation occurs and no packet loss occurs, the receiving apparatus 200 stores in the MMT packet based on the number of slice segments acquired after detecting the head of the access unit. The index number of the slice segment can be specified.

Similarly, when the data unit of the data before the slice segment is fragmented, the receiving apparatus 200 can similarly detect the access unit and the head of the slice segment.

In addition, when packet loss occurs or when SPS, PPS, and SEI included in the data before the slice segment are set to different Data units, the receiving apparatus 200 slices based on the analysis result of the MMT header. The start position of the slice segment or tile in the picture (access unit) can be specified by specifying the MMT packet storing the head data of the segment and then analyzing the header of the slice segment. The amount of processing related to the analysis of the slice header is small, and the processing load does not matter.

Thus, each of a plurality of encoded data of a plurality of slice segments is associated with a basic data unit (Data unit) which is a unit of data stored in one or more packets on a one-to-one basis. Each of the plurality of encoded data is stored in one or more MMT packets.

The header information of each MMT packet includes a fragmentation indicator (identification information) and an Offset (offset information).

The receiving apparatus 200 determines that the beginning of the payload data included in the packet having the header information including the fragmentation indicator whose value is 00 or 01 is the beginning of the encoded data of each slice segment. . Specifically, the head of the payload data included in the packet having header information including the offset that is not 0 and the fragmentation indicator that is 00 or 01 is the head of the encoded data of each slice segment. Is determined.

In the example of FIG. 17, the beginning of the Data unit is either the beginning of the access unit or the beginning of the slice segment, and the value of the fragmentation indicator is 00 or 01. Further, the receiving apparatus 200 refers to the type of the NAL unit, determines whether the head of the Data Unit is an access unit delimiter or a slice segment, and without referring to the Offset, It is also possible to detect the head or the head of a slice segment.

In this way, including the case where the transmission apparatus 100 performs packetization so that the head of the NAL unit always starts from the head of the payload of the MMT packet, so that the data before the slice segment is divided into a plurality of data units. The receiving apparatus 200 can detect the head of the access unit or the slice segment by analyzing the fragmentation indicator and the NAL unit header. The type of the NAL unit is present in the first byte of the NAL unit header. Therefore, when analyzing the header part of the MMT packet, the receiving apparatus 200 can acquire the type of the NAL unit by additionally analyzing the data for 1 byte.

In the case of audio, the receiving apparatus 200 only needs to be able to detect the head of the access unit, and may determine based on whether the value of the fragmentation indicator is 00 or 01.

Further, as described above, when the encoded data encoded so as to be divided and decoded is stored in the PES packet of the MPEG-2 TS, the transmission apparatus 100 can use the data alignment descriptor. is there. Hereinafter, an example of a method for storing encoded data in a PES packet will be described in detail.

For example, in HEVC, the transmission apparatus 100 can indicate whether the data stored in the PES packet is an access unit, a slice segment, or a tile by using the data alignment descriptor. The alignment type in HEVC is defined as follows.

Alignment type = 8 indicates HEVC slice segment. Alignment type = 9 indicates a HEVC slice segment or access unit. Alignment type = 12 indicates a HEVC slice segment or tile.

Therefore, the transmission device 100 can indicate that the data of the PES packet is either the slice segment or the data before the slice segment by using, for example, type 9. Since a type indicating a slice instead of a slice segment is separately defined, the transmission apparatus 100 may use a type indicating a slice instead of a slice segment.

Also, the DTS and PTS included in the header of the PES packet are set only in the PES packet including the head data of the access unit. Therefore, if the type is 9 and the DTS or PTS field is present in the PES packet, the receiving apparatus 200 stores the entire access unit or the first division unit in the access unit in the PES packet. Can be judged.

In addition, the transmission apparatus 100 may use a field such as transport_priority indicating the priority of a TS packet that stores a PES packet including the head data of the access unit so that the reception apparatus 200 can distinguish data included in the packet. Good. The receiving apparatus 200 may determine data included in the packet by analyzing whether the payload of the PES packet is an access unit delimiter. Further, data_alignment_indicator of the PES packet header indicates whether data is stored in the PES packet according to these types. If this flag (data_alignment_indicator) is set to 1, it is guaranteed that the data stored in the PES packet conforms to the type indicated in the data alignment descriptor.

Further, the transmission apparatus 100 may use the data alignment descriptor only when PES packets are generated in units that can be divided and decoded such as slice segments. As a result, if there is a data alignment descriptor, the receiving apparatus 200 can determine that the encoded data is PES packetized in a unit that can be divided and decoded, and if there is no data alignment descriptor, the receiving apparatus 200 It can be determined that the digitized data is PES packetized in units of access units. It should be noted that when data_alignment_indicator is set to 1 and there is no data alignment descriptor, it is specified in the MPEG-2 TS standard that the unit of PES packetization is an access unit.

If the data alignment descriptor is included in the PMT, the receiving apparatus 200 determines that the PES packet is generated in a unit that can be divided and decoded, and inputs to each decoding unit based on the packetized unit. Data can be generated. Further, when the data alignment descriptor is not included in the PMT, the receiving apparatus 200 determines that parallel decoding of encoded data is necessary based on program information or other descriptor information. By analyzing the slice header of the slice segment, input data to each decoding unit is generated. In addition, when the encoded data can be decoded by a single decoding unit, receiving apparatus 200 decodes the data of the entire access unit by the decoding unit. Note that if the information indicating whether the encoded data is composed of units that can be divided and decoded, such as slice segments or tiles, is separately indicated by a PMT descriptor or the like, the receiving apparatus 200 may Whether the encoded data can be decoded in parallel may be determined based on the analysis result.

In addition, since the DTS and PTS included in the header of the PES packet are set only in the PES packet including the head data of the access unit, when the access unit is divided into PES packets, the second and subsequent PESs are set. The packet does not include information indicating the DTS and PTS of the access unit. Therefore, when performing the decoding process in parallel, each of the decoding units 204A to 204D and the display unit 205 uses the DTS and PTS stored in the header of the PES packet including the head data of the access unit.

(Embodiment 2)
In the second embodiment, a method for storing NAL size format data in an MP4 format based MPU in MMT will be described. In the following, a storage method in an MPU used for MMT will be described as an example, but such a storage method can be applied to DASH that is based on the same MP4 format.

[How to store in MPU]
In the MP4 format, a plurality of access units are collected and stored in one MP4 file. In the MPU used for MMT, data for each medium is stored in one MP4 file, and the data can include any number of access units. Since the MPU is a unit that can be decoded by itself, for example, an MPU-based access unit is stored in the MPU.

FIG. 19 is a diagram showing the configuration of the MPU. The head of MPU is ftyp, mmpu, and moov, and these are collectively defined as MPU metadata. The moov stores initialization information common to files and an MMT hint track.

The moof includes initialization information and size for each sample and subsample, information (sample_duration, sample_size, sample_composition_time_offset) that can specify the presentation time (PTS) and the decoding time (DTS), and data_offset indicating the position of the data. Stored.

Further, each of the plurality of access units is stored as a sample in mdat (mdat box). The data excluding samples in moof and mdat is defined as movie fragment metadata (hereinafter referred to as MF metadata), and the mdat sample data is defined as media data.

FIG. 20 is a diagram showing the structure of MF metadata. As shown in FIG. 20, the MF metadata is more specifically composed of a type, length, and data of a moof box (moof), and a type and length of an mdat box (mdat).

When storing the access unit in MP4 data, H.264 and H.264. There are a mode in which a parameter set such as S265 and PPS of H.265 can be stored as sample data, and a mode in which the parameter set cannot be stored.

Here, in the mode that cannot be stored, the parameter set is stored in the Decoder Specific Information of SampleEntry in moov. Moreover, in the mode that can be stored, the parameter set is included in the sample.

MPU metadata, MF metadata, and media data are each stored in the MMT payload, and a fragment type (FT) is stored in the header of the MMT payload as an identifier that can identify these data. FT = 0 indicates MPU metadata, FT = 1 indicates MF metadata, and FT = 2 indicates media data.

Note that FIG. 19 illustrates an example in which the MPU metadata unit and the MF metadata unit are stored as data units in the MMT payload. However, units such as ftyp, mmpu, moov, and moof are data units. It may be stored in the MMT payload in units. Similarly, FIG. 19 illustrates an example in which a sample unit is stored as a data unit in the MMT payload. However, data units may be configured in units of samples or NAL units, and such data units may be stored in the MMT payload in units of data units. Such data units may be stored in the MMT payload in units that are further fragmented.

[Conventional transmission methods and issues]
Conventionally, when a plurality of access units are encapsulated in MP4 format, moov and moof are created when all the samples stored in MP4 are prepared.

When transmitting the MP4 format in real time using broadcasting or the like, for example, assuming that samples stored in one MP4 file are in GOP units, moov and moof are created after time samples in GOP units are accumulated. There is a delay associated with encapsulation. By such encapsulation on the transmission side, the end-to-end delay is always increased by the GOP unit time. Accordingly, it becomes difficult to provide a service in real time, and particularly when live content is transmitted, the service for the viewer is deteriorated.

FIG. 21 is a diagram for explaining the data transmission order. When MMT is applied to broadcasting, as shown in FIG. 21A, transmission is carried on MMT packets in the order of MPU configuration (MMT packets # 1, # 2, # 3, # 4, # 5, # 6). Transmission in this order), a delay due to encapsulation occurs in the transmission of the MMT packet.

In order to prevent this delay due to encapsulation, as shown in FIG. 21B, MPU header information such as MPU metadata and MF metadata is not transmitted (packets # 1 and # 2 are not transmitted, packet # 3- # 6 is transmitted in this order). Also, as shown in FIG. 20C, the media data is transmitted first without waiting for the creation of the MPU header information, and the MPU header information is transmitted after the media data is transmitted (# 3- # 6, # 1 and # 2 are transmitted in this order).

When the MPU header information is not transmitted, the receiving device performs decoding without using the MPU header information. When the MPU header information is postponed with respect to the media data, the receiving device decodes the MPU header information. Decrypt after waiting for information to be acquired.

However, a conventional MP4 compliant receiving device is not guaranteed to decode without using MPU header information. In addition, when the receiving apparatus performs decoding without using the MPU header by special processing, if the conventional transmission method is used, the decoding processing becomes complicated, and real-time decoding is likely to be difficult. In addition, when decoding is performed after the receiving apparatus waits for acquisition of MPU header information, buffering of media data is necessary until the receiving apparatus acquires header information, but a buffer model is not defined. Decryption was not guaranteed.

Therefore, as shown in FIG. 20D, the transmitting apparatus according to Embodiment 2 transmits MPU metadata earlier than the media data by storing only common information in the MPU metadata. Then, the transmission apparatus according to Embodiment 2 transmits the MF metadata that is delayed in generation after the media data. This provides a transmission method or reception method that can guarantee decoding of media data.

Hereinafter, a reception method when each of the transmission methods (a) to (d) in FIG. 21 is used will be described.

In each transmission method shown in FIG. 21, first, MPU data is configured in the order of MPU metadata, MFU metadata, and media data.

After the MPU data is configured, when the transmitting device transmits data in the order of MPU metadata, MF metadata, and media data as shown in FIG. 21A, the receiving device has the following (A-1 ) And (A-2) can be used for decoding.

(A-1) After receiving the MPU header information (MPU metadata and MF metadata), the receiving device decrypts the media data using the MPU header information.

(A-2) The receiving device decodes the media data without using the MPU header information.

Any of these methods has an advantage in that there is a delay due to encapsulation on the transmission side, but there is no need to buffer media data in order to obtain the MPU header in the receiving apparatus. When buffering is not performed, it is not necessary to install a memory for buffering, and no buffering delay occurs. Further, since the method (A-1) performs decoding using MPU header information, it can also be applied to a conventional receiving apparatus.

When the transmitting device transmits only media data as shown in FIG. 21B, the receiving device can perform decoding by the following method (B-1).

(B-1) The receiving device decodes the media data without using the MPU header information.

Although not shown, when the MPU metadata is transmitted prior to the transmission of the media data in FIG. 21B, the decoding can be performed by the following method (B-2).

(B-2) The receiving device decrypts the media data using the MPU metadata.

Both the methods (B-1) and (B-2) are advantageous in that no delay is caused by encapsulation on the transmission side, and there is no need to buffer media data for obtaining the MPU header. It is. However, since the methods (B-1) and (B-2) do not perform decoding using the MPU header information, special processing may be required for decoding.

When the transmitting apparatus transmits data in the order of media data, MPU metadata, and MF metadata as shown in FIG. 21 (c), the receiving apparatus performs the following (C-1) and (C-2) Decoding can be performed by either method.

(C-1) The receiving device decodes the media data after obtaining the MPU header information (MPU metadata and MF metadata).

(C-2) The receiving device decodes the media data without using the MPU header information.

When the above method (C-1) is used, it is necessary to buffer the media data for obtaining the MPU header information. On the other hand, when the method (C-2) is used, it is not necessary to perform buffering for obtaining MPU header information.

Also, in any of the methods (C-1) and (C-2), there is no delay due to encapsulation on the transmission side. In addition, since the method (C-2) does not use MPU header information, special processing may be required.

When the transmitting apparatus transmits data in the order of MPU metadata, media data, and MF metadata, as shown in FIG. 21 (d), the receiving apparatus performs the following (D-1) and (D-2) ) Can be decrypted by any of the methods.

(D-1) After receiving the MPU metadata, the receiving device further acquires MF metadata, and then decrypts the media data.

(D-2) After receiving the MPU metadata, the receiving device decrypts the media data without using the MF metadata.

When the method (D-1) is used, it is necessary to buffer the media data for obtaining the MF metadata. In the case of the method (D-2), for obtaining the MF metadata. There is no need to buffer.

Since the method (D-2) does not perform decoding using MF metadata, special processing may be required.

As described above, when decoding can be performed using MPU metadata and MF metadata, there is a merit that the conventional MP4 receiving apparatus can also perform decoding.

In FIG. 21, the MPU data is configured in the order of MPU metadata, MFU metadata, and media data. In the moof, position information (offset) for each sample or subsample is determined based on this configuration. ing. The MF metadata includes data (box size and type) other than the media data in the mdat box.

For this reason, when the receiving device specifies media data based on the MF metadata, the receiving device reconfigures the data in the order in which the MPU data is configured, regardless of the order in which the data is transmitted. , Decoding is performed using the moov of MPU metadata or the moof of MF metadata.

In FIG. 21, MPU data is configured in the order of MPU metadata, MFU metadata, and media data. However, MPU data is configured in an order different from that in FIG. 21, and position information (offset) is determined. Good.

For example, MPU data may be configured in the order of MPU metadata, media data, and MF metadata, and negative position information (offset) may be indicated in the MF metadata. In this case as well, regardless of the order in which data is transmitted, the receiving apparatus reconstructs data in the order in which MPU data is configured on the transmission side, and then performs decoding using moov or moof.

Note that the transmission device may signal information indicating the order in which the MPU data is configured, and the reception device may reconfigure the data based on the signaled information.

As described above, the receiving apparatus receives packetized MPU metadata, packetized media data (sample data), and packetized MF metadata as shown in FIG. Receive in order. Here, the MPU metadata is an example of first metadata, and the MF metadata is an example of second metadata.

Next, the receiving apparatus reconstructs MPU data (MP4 format file) including the received MPU metadata, the received MF metadata, and the received sample data. Then, the sample data included in the reconstructed MPU data is decoded using the MPU metadata and the MF metadata. The MF metadata is metadata including data (for example, length stored in mbox) that can be generated only after sample data is generated on the transmission side.

Note that the operation of the receiving apparatus is performed in more detail by each component constituting the receiving apparatus. For example, the reception device includes a reception unit that receives the data, a reconstruction unit that reconstructs the MPU data, and a decoding unit that decodes the MPU data. Each of the reception unit, the generation unit, and the decoding unit is realized by a microcomputer, a processor, a dedicated circuit, and the like.

[Decoding without using header information]
Next, a method for performing decoding without using header information will be described. Here, a method of decoding without using header information in the receiving apparatus regardless of whether header information is sent or not on the transmission side will be described. That is, this method can be applied to any of the transmission methods described with reference to FIG. However, some decoding methods are decoding methods applicable only to a specific transmission method.

FIG. 22 is a diagram illustrating an example of a method for performing decoding without using header information. In FIG. 22, only the MMT payload and MMT packet including only media data are illustrated, and the MMT payload and MMT packet including MPU metadata and MF metadata are not illustrated. In the description of FIG. 22 below, it is assumed that media data belonging to the same MPU is continuously transmitted. Further, a case where a sample is stored in the payload as media data will be described as an example. However, in the following description of FIG. 22, a NAL unit may naturally be stored, or a fragmented NAL unit is stored. It may be.

In order to decrypt media data, the receiving device must first acquire initialization information necessary for decryption. If the media is video, the receiving apparatus must acquire initialization information for each sample, specify the start position of the MPU that is a random access unit, and acquire the start position of the sample and the NAL unit. I must. In addition, the receiving device needs to specify the decoding time (DTS) and presentation time (PTS) of each sample.

Therefore, the receiving apparatus can perform decoding without using header information using the following method, for example. In the case where a NAL unit unit or a unit obtained by fragmenting a NAL unit is stored in the payload, “sample” in the following description may be read as “NAL unit in sample”.

<Random access (= specify first sample of MPU)>
When the header information is not transmitted, there are the following method 1 and method 2 for the receiving apparatus to specify the head sample of the MPU. Note that Method 3 can be used when header information is transmitted.

[Method 1] The receiving apparatus acquires a sample included in the MMT packet with ‘RAP_flag = 1’ in the MMT packet header.

[Method 2] The receiving apparatus obtains a sample having “sample number = 0” in the MMT payload header.

[Method 3] When at least one of MPU metadata and MF metadata is transmitted to at least one of the front and rear of the media data, the receiving device transmits the fragment type in the MMT payload header. A sample included in the MMT payload in which (FT) is switched to media data is acquired.

In the method 1 and the method 2, when a plurality of samples belonging to different MPUs coexist in one payload, it is impossible to determine which NAL unit is a random access point (RAP_flag = 1 or sample number = 0). For this reason, in the case where different MPU samples are not mixed in one payload, or when different MPU samples are mixed in one payload, the RAP_flag is set when the last (or first) sample is a random access point. A constraint such as 1 is necessary.

Also, in order for the receiving apparatus to acquire the start position of the NAL unit, it is necessary to shift the data read pointer by the size of the NAL unit in order from the head NAL unit of the sample.

When the data is fragmented, the receiving device can identify the data unit by referring to the fragment_indicator and the fragment_number.

<DTS determination of sample>
The method for determining the DTS of the sample includes the following method 1 and method 2.

[Method 1] The receiving apparatus determines the DTS of the first sample based on the prediction structure. However, since this method requires analysis of encoded data and may be difficult to decode in real time, the following method 2 is desirable.

[Method 2] The receiving apparatus separately transmits the DTS of the first sample, and acquires the transmitted DTS of the first sample. As a method for transmitting the DTS of the first sample, there are, for example, a method of transmitting the DTS of the MPU first sample using MMT-SI, a method of transmitting the DTS of each sample using the MMT packet header extension area, and the like. The DTS may be an absolute value or a relative value with respect to the PTS. Further, it may be signaled on the transmission side whether or not the DTS of the first sample is included.

In both

methods

1 and 2, the DTS of the subsequent samples is calculated as a fixed frame rate.

As a method of storing the DTS for each sample in the packet header, there is a method of storing the DTS of the sample included in the MMT packet in the 32-bit NTP timestamp field in the MMT packet header, in addition to using the extension area. When the DTS cannot be expressed by the number of bits (32 bits) of one packet header, the DTS may be expressed using a plurality of packet headers. Further, the DTS may be expressed by combining the NTP timestamp field of the packet header and the extension area. When the DTS information is not included, a known value (for example, ALL0) is set.

<Determining the PTS of the sample>
The receiving device acquires the PTS of the first sample from the MPU time stamp descriptor for each asset included in the MPU. The receiving apparatus calculates the subsequent sample PTS as a fixed frame rate from a parameter indicating the display order of the samples such as POC. Thus, in order to calculate DTS and PTS without using header information, transmission at a fixed frame rate is essential.

When the MF metadata is transmitted, the receiving apparatus transmits the relative time information of the DTS and PTS from the first sample indicated in the MF metadata and the time stamp of the MPU first sample indicated in the MPU time stamp descriptor. The absolute value of DTS and PTS can be calculated from the absolute value.

Note that when calculating the DTS and PTS by analyzing the encoded data, the receiving apparatus may calculate using the SEI information included in the access unit.

<Initialization information (parameter set)>
[For video]
In the case of video, the parameter set is stored in sample data. In addition, when MPU metadata and MF metadata are not transmitted, it is ensured that a parameter set necessary for decoding can be acquired by referring to only sample data.

Further, as shown in FIGS. 21A and 21D, when MPU metadata is transmitted before media data, it may be specified that no parameter set is stored in SampleEntry. In this case, the receiving apparatus refers to only the parameter set in the sample without referring to the parameter set of SampleEntry.

When MPU metadata is transmitted before media data, SampleEntry stores a parameter set common to the MPU and a default parameter set, and the receiving apparatus stores the parameter set of SampleEntry and the parameter set in the sample. You may refer to it. By storing the parameter set in SampleEntry, it is possible to perform decoding even in a conventional receiving apparatus that cannot be reproduced unless the parameter set exists in SampleEntry.

[For audio]
In the case of audio, the LATM header is required for decoding, and in MP4, it is essential that the LATM header be included in the sample entry. However, when the header information is not transmitted, it is difficult for the receiving apparatus to obtain the LATM header, and therefore the LATM header is included in control information such as SI separately. Note that the LATM header may be included in a message, table, or descriptor. Note that the LATM header may be included in the sample.

The receiving device acquires the LATM header from the SI or the like before starting decoding, and starts decoding audio. Alternatively, as shown in FIGS. 21A and 21D, when the MPU metadata is transmitted before the media data, the receiving device receives the LATM header before the media data. Is possible. Therefore, when MPU metadata is transmitted before media data, it is possible to perform decoding using a conventional receiving apparatus.

<Others>
The transmission order and the type of transmission order may be notified as control information such as an MMT packet header, a payload header, or an MPT, other tables, messages, and descriptors. Note that the types of transmission order here are, for example, the transmission types of the four types (a) to (d) in FIG. 21, and identifiers for identifying the respective types can be acquired before starting decoding. What is necessary is just to be stored in a place.

Also, as the transmission order type, different types may be used for audio and video, or a common type may be used for audio and video. Specifically, for example, the audio is transmitted in the order of MPU metadata, MF metadata, and media data as shown in FIG. 21A, and the video is shown in FIG. 21D. Thus, MPU metadata, media data, and MF metadata may be transmitted in this order.

By the method as described above, the receiving device can perform decoding without using header information. Further, when the MPU metadata is transmitted before the media data (FIG. 21 (a) and FIG. 21 (d)), it is possible to perform decoding also by a conventional receiving apparatus.

In particular, since the MF metadata is transmitted after the media data ((d) in FIG. 21), it is possible to perform decoding even in a conventional receiving apparatus without causing a delay due to encapsulation.

[Configuration and operation of transmitter]
Next, the configuration and operation of the transmission apparatus will be described. FIG. 23 is a block diagram of a transmission apparatus according to the second embodiment, and FIG. 24 is a flowchart of the transmission method according to the second embodiment.

23, the transmission device 15 includes an encoding unit 16, a multiplexing unit 17, and a transmission unit 18.

The encoding unit 16 converts the video or audio to be encoded into, for example, H.264. Encoded data is generated by encoding according to H.265 (S10).

The multiplexing unit 17 multiplexes (packets) the encoded data generated by the encoding unit 16 (S11). Specifically, the multiplexing unit 17 packetizes each of sample data, MPU metadata, and MF metadata that make up an MP4 format file. The sample data is data in which a video signal or an audio signal is encoded, MPU metadata is an example of first metadata, and MF metadata is an example of second metadata. Both the first metadata and the second metadata are metadata used for decoding the sample data, but the difference between them is data that can be generated only after the second metadata generates the sample data. It is to include.

Here, data that can be generated only after generation of sample data is, for example, data other than the sample data stored in mdat in the MP4 format (data in the header of mdat. That is, type and length shown in FIG. 20). ). Here, the second metadata only needs to include a length which is at least a part of this data.

The transmitting unit 18 transmits the packetized MP4 format file (S12). For example, the transmission unit 18 transmits the MP4 format file by the method shown in FIG. That is, the packetized MPU metadata, the packetized sample data, and the packetized MF metadata are transmitted in this order.

Note that each of the encoding unit 16, the multiplexing unit 17, and the transmission unit 18 is realized by a microcomputer, a processor, a dedicated circuit, or the like.

[Receiver configuration]
Next, the configuration and operation of the receiving apparatus will be described. FIG. 25 is a block diagram of a receiving apparatus according to the second embodiment.

As illustrated in FIG. 25, the reception device 20 includes a packet filtering unit 21, a transmission order type determination unit 22, a random access unit 23, a control information acquisition unit 24, a data acquisition unit 25, a PTS and a DTS calculation. Unit 26, initialization information acquisition unit 27, decryption command unit 28, decryption unit 29, and presentation unit 30.

[Operation 1 of receiving apparatus]
First, an operation for the receiving device 20 to specify the MPU head position and the NAL unit position when the medium is video will be described. FIG. 26 is a flowchart of such an operation of the receiving device 20. Here, it is assumed that the transmission order type of MPU data is stored in the SI information by the transmission device 15 (multiplexer 17).

First, the packet filtering unit 21 performs packet filtering on the received file. The transmission order type determination unit 22 analyzes the SI information obtained by packet filtering and acquires the transmission order type of MPU data (S21).

Next, the transmission order type determination unit 22 determines (determines) whether or not MPU header information (at least one of MPU metadata or MF metadata) is included in the data after packet filtering (S22). When the MPU header information is included (Yes in S22), the random access unit 23 identifies the MPU head sample by detecting that the fragment type of the MMT payload header is switched to the media data (S23). ).

On the other hand, when the MPU header information is not included (No in S22), the random access unit 23 specifies the MPU head sample based on the RAP_flag of the MMT packet header or the sample number of the MMT payload header (S24).

Further, the transmission order type determination unit 22 determines whether or not MF metadata is included in the packet filtered data (S25). If it is determined that MF metadata is included (Yes in S25), the data acquisition unit 25 reads the NAL unit based on the sample, subsample offset, and size information included in the MF metadata. As a result, the NAL unit is acquired (S26). On the other hand, when it is determined that the MF metadata is not included (No in S25), the data acquisition unit 25 reads the NAL unit size data in order from the first NAL unit of the sample. Obtain (S27).

Note that, even when it is determined in step S22 that the MPU header information is included, the reception device 20 may specify the MPU head sample by using the process of step S24 instead of step S23. Further, when it is determined that the MPU header information is included, the process of step S23 and the process of step S24 may be used in combination.

Further, even when it is determined in step S25 that the MF metadata is included, the reception device 20 may acquire the NAL unit using the process of step S27 without using the process of step S26. Further, when it is determined that the MF metadata is included, the process of step S23 and the process of step S24 may be used in combination.

Also, it is assumed that it is determined in step S25 that MF metadata is included, and the MF data is transmitted after the media data. In this case, the receiving device 20 may buffer the media data and wait for the acquisition of MF metadata before performing the process of step S26. The receiving device 20 does not wait for the acquisition of MF metadata. It may be determined whether or not the process of step S27 is performed.

For example, the receiving apparatus 20 may determine whether to wait for acquisition of MF metadata based on whether or not the buffer of a buffer size capable of buffering media data is held. Further, the reception device 20 may determine whether to wait for acquisition of MF metadata based on whether the End-to-End delay is small. The receiving apparatus 20 may perform the decoding process mainly using the process of step S26, and may use the process of step S27 in the processing mode when packet loss or the like occurs.

If the transmission order type is determined in advance, step S22 and step S26 may be omitted. In this case, the receiving apparatus 20 considers the buffer size and the end-to-end delay, You may determine the identification method of a head sample, and the identification method of a NAL unit.

If the transmission order type is known in advance, the transmission order type determination unit 22 in the receiving device 20 is not necessary.

Although not described in FIG. 26, the decoding instruction unit 28 obtains data based on the PTS and DTS calculated in the PTS and DTS calculation unit 26, and the initialization information acquired in the initialization information acquisition unit 27. The data acquired in the unit is output to the decoding unit 29. The decryption unit 29 decrypts the data, and the presentation unit 30 presents the decrypted data.

[Operation 2 of receiving apparatus]
Next, an operation in which the receiving device 20 acquires initialization information based on the transmission order type and decodes media data based on the initialization information will be described. FIG. 27 is a flowchart of such an operation.

First, the packet filtering unit 21 performs packet filtering on the received file. The transmission order type determination unit 22 analyzes SI information obtained by packet filtering and acquires a transmission order type (S301).

Next, the transmission order type determination unit 22 determines whether MPU metadata is transmitted (S302). When it is determined that the MPU metadata is transmitted (Yes in S302), the transmission order type determination unit 22 determines whether the MPU metadata is transmitted before the media data as a result of the analysis in step S301. (S303). When the MPU metadata is transmitted before the media data (Yes in S303), the initialization information acquisition unit 27 is based on the common initialization information included in the MPU metadata and the initialization information of the sample data. The media data is decrypted (S304).

On the other hand, when it is determined that the MPU metadata is transmitted after the media data (No in S303), the data acquisition unit 25 buffers the media data until the MPU metadata is acquired (S305). After the MPU metadata is acquired, the process of step S304 is performed.

If it is determined in step S302 that the MPU metadata is not transmitted (No in S302), the initialization information acquisition unit 27 decodes the media data based only on the initialization information of the sample data. (S306).

If the decoding of the media data is guaranteed only when the transmission side is based on the initialization information of the sample data, the process of step S306 is used without performing the process based on the determinations of steps S302 and S303.

Further, the receiving device 20 may determine whether or not to buffer the media data before step S305. In this case, if it is determined that the media data is to be buffered, the receiving apparatus 20 proceeds to the process of step S305. If it is determined that the media data is not to be buffered, the receiving apparatus 20 proceeds to the process of step S306. The determination as to whether or not to buffer the media data may be made based on the buffer size and occupation amount of the receiving apparatus 20, or, for example, the end-to-end delay having a smaller end is selected. The determination may be made taking into account the -to-End delay.

[Operation 3 of receiving apparatus]
Here, the details of the transmission method and the reception method when the MF metadata is transmitted after the media data ((c) in FIG. 21 and (d) in FIG. 21) will be described. Hereinafter, the case of FIG. 21D will be described as an example. In transmission, only the method of FIG. 21D is used, and transmission order type signaling is not performed.

As described above, as shown in FIG. 21D, when data is transmitted in the order of MPU metadata, media data, and MF metadata,
(D-1) The receiving apparatus 20 decodes the media data after obtaining MPU metadata and further obtaining MF metadata.

(D-2) After receiving the MPU metadata, the receiving device 20 decodes the media data without using the MF metadata.
The following two decoding methods are possible.

Here, D-1 requires buffering of media data for MF metadata acquisition, but can be decoded using MPU header information, so that it can be decoded by a conventional MP4-compliant receiver. It becomes. D-2 does not require buffering of media data for obtaining MF metadata, but cannot be decrypted using MF metadata, and thus requires special processing for decoding.

Further, the method of (d) in FIG. 21 has an advantage that since the MF metadata is transmitted after the media data, there is no delay due to encapsulation, and the end-to-end delay can be reduced.

The receiving device 20 can select the above two decoding methods according to the capability of the receiving device 20 and the service quality provided by the receiving device 20.

The transmission device 15 must ensure that decoding can be performed while reducing the occurrence of buffer overflow and underflow in the decoding operation of the reception device 20. For example, the following parameters can be used as elements for defining a decoder model for decoding using the method D-1.

-Buffer size for reconfiguring the MPU (MPU buffer)
For example, buffer size = maximum rate × maximum MPU time × α, and the maximum rate is the profile of encoded data, the upper limit rate of level + MPU header overhead. The maximum MPU time is the maximum time length of a GOP when 1 MPU = 1 GOP (video).

Here, the audio may be a GOP unit common to the video, or may be another unit. α is a margin for preventing overflow, and may be multiplied or added to the maximum rate × maximum MPU time. When multiplying, α ≧ 1, and when adding, α ≧ 0.

-Upper limit of decoding delay time from data input to MPU buffer until decoding (TSTD_delay in STD of MPEG-TS)
For example, at the time of transmission, the DTS is set so that the MPU data acquisition completion time <= DTS in the receiver in consideration of the maximum MPU time and the upper limit of the decoding delay time.

Further, the transmission device 15 may add DTS and PTS according to a decoder model in the case of decoding using the method D-1. As a result, the transmitting device 15 guarantees the decoding of the receiving device that performs decoding using the method D-1, and simultaneously transmits auxiliary information required when decoding is performed using the method D-2. May be.

For example, the transmitter 15 can guarantee the operation of the receiver that decodes using the D-2 method by signaling the pre-buffering time in the decoder buffer when decoding using the D-2 method.

The pre-buffering time may be included in SI control information such as a message, a table, or a descriptor, or may be included in the header of an MMT packet or MMT payload. Further, the SEI in the encoded data may be overwritten. The DTS and PTS for decoding using the method D-1 are stored in the MPU time stamp descriptor, SampleEntry, and the DTS and PTS for decoding using the method D-2, or the pre-buffering time. It may be described in SEI.

When the receiving device 20 supports only the MP4-compliant decoding operation using the MPU header, the receiving device 20 selects the decoding method D-1 and supports both D-1 and D-2. If so, either one may be selected.

The transmission device 15 may add DTS and PTS so as to guarantee one (D-1 in this description) decoding operation, and may transmit auxiliary information for assisting one decoding operation.

Further, when the method D-2 is used, the end-to-end delay may be increased due to the delay caused by the pre-buffering of the MF metadata as compared with the case where the method D-1 is used. Is expensive. Therefore, the receiving apparatus 20 may select and decode the D-2 method when it is desired to reduce the End-to-End delay. For example, the receiving apparatus 20 may always use the D-2 method when it is desired to always reduce the end-to-end delay. In addition, the receiving apparatus 20 may use the method D-2 only when it is operating in the low delay presentation mode, such as live content, channel selection, zapping operation, etc.

FIG. 28 is a flowchart of such a receiving method.

First, the receiving device 20 receives the MMT packet and acquires MPU data (S401). Then, the receiving device 20 (transmission order type determining unit 22) determines whether or not to present the program in the low-delay presentation mode (S402).

When the program is not presented in the low-delay presentation mode (No in S402), the reception device 20 (random access unit 23 and initialization information acquisition unit 27) acquires random access and initialization information using the header information (S405). ). The receiving device 20 (PTS, DTS calculation unit 26, decoding command unit 28, decoding unit 29, and presentation unit 30) performs decoding and presentation processing based on the PTS and DTS assigned on the transmission side (S406).

On the other hand, when the program is presented in the low-delay presentation mode (Yes in S402), the receiving device 20 (random access unit 23 and initialization information acquisition unit 27) uses a decoding method that does not use header information, random access, Initialization information is acquired (S403). Further, the receiving device 20 performs decoding and presentation processing based on auxiliary information for decoding without using the PTS, DTS, and header information given on the transmission side (S404). In step S403 and step S404, processing may be performed using MPU metadata.

[Transmission / reception method using auxiliary data]
The transmission / reception operation when the MF metadata is transmitted after the media data (in the case of (c) in FIG. 21 and in the case of (d) in FIG. 21) has been described above. Next, a description will be given of a method in which the transmission device 15 can start decoding earlier by transmitting auxiliary data having a partial function of MF metadata, and can reduce the End-to-End delay. Here, an example in which auxiliary data is further transmitted based on the transmission method shown in (d) of FIG. 21 will be described. The method using auxiliary data is shown in (a) to (c) of FIG. The present invention can also be applied to other transmission methods.

(A) of FIG. 29 is a diagram showing an MMT packet transmitted using the method shown in (d) of FIG. That is, the data is transmitted in the order of MPU metadata, media data, and MF metadata.

Here, sample # 1, sample # 2, sample # 3, and sample # 4 are samples included in the media data. Although an example in which media data is stored in an MMT packet in units of samples will be described here, media data may be stored in MMT packets in units of NAL units, or stored in units obtained by dividing NAL units. May be. A plurality of NAL units may be aggregated and stored in the MMT packet.

As described in D-1 above, in the case of the method shown in FIG. 21D, that is, when data is transmitted in the order of MPU metadata, media data, and MF metadata, MPU metadata is There is a method of acquiring MF metadata after acquisition and then decoding the media data. Such a method of D-1 requires buffering of media data for obtaining MF metadata. However, since decoding is performed using MPU header information, a conventional MP4-compliant receiving apparatus can also perform D-reading. The method -1 has the advantage of being applicable. On the other hand, there is a drawback that the receiving device 20 has to wait for decoding to start until MF metadata acquisition.

On the other hand, as shown in FIG. 29B, in the method using auxiliary data, auxiliary data is transmitted prior to MF metadata.

MF metadata includes information indicating DTS and PTS, offset, and size of all samples included in the movie fragment. On the other hand, the auxiliary data includes information indicating the DTS and PTS, offset, and size of some of the samples included in the movie fragment.

For example, the MF metadata includes information on all samples (sample # 1-sample # 4), whereas the auxiliary data includes information on some samples (sample # 1- # 2). .

In the case shown in (b) of FIG. 29, since the auxiliary data is used, the samples # 1 and # 2 can be decoded. Delay is reduced. The auxiliary data may include sample information in any combination, and the auxiliary data may be repeatedly transmitted.

For example, in FIG. 29C, when transmitting auxiliary information at the timing A, the transmission device 15 includes the information of sample # 1 in the auxiliary information, and when transmitting auxiliary information at the timing B, Information on sample # 1 and sample # 2 is included in the auxiliary information. When transmitting the auxiliary information at timing C, the transmission device 15 includes the information of sample # 1, sample # 2, and sample # 3 in the auxiliary information.

The MF metadata includes information on sample # 1, sample # 2, sample # 3, and sample # 4 (information on all samples in the movie fragment).

Assistance data does not necessarily need to be sent immediately after generation.

In the header of the MMT packet or MMT payload, a type indicating that auxiliary data is stored is designated.

For example, when auxiliary data is stored in the MMT payload using the MPU mode, a data type indicating auxiliary data is specified as a fragment_type field value (for example, FT = 3). The auxiliary data may be data based on the structure of moof, or may have other structures.

When the auxiliary data is stored as a control signal (descriptor, table, message) in the MMT payload, a descriptor tag indicating that it is auxiliary data, a table ID, a message ID, and the like are specified.

Also, PTS or DTS may be stored in the header of the MMT packet or MMT payload.

[Example of auxiliary data generation]
Hereinafter, an example in which the transmission apparatus generates auxiliary data based on the configuration of moof will be described. FIG. 30 is a diagram for explaining an example in which the transmission apparatus generates auxiliary data based on the moof configuration.

In normal MP4, as shown in FIG. 20, a moof is created for a movie fragment. The “moof” includes DTS and PTS of the sample included in the movie fragment, and information indicating the offset and size.

Here, the transmission device 15 configures MP4 (MP4 file) using only a part of sample data among the sample data configuring the MPU, and generates auxiliary data.

For example, as illustrated in FIG. 30A, the transmission device 15 generates MP4 using only the sample # 1 among the samples # 1 to # 4 constituting the MPU, and among them, the header of moof + mdat is generated. Use auxiliary data.

Next, as illustrated in (b) of FIG. 30, the transmission device 15 generates MP4 using the sample # 1 and the sample # 2 among the samples # 1 to # 4 constituting the MPU, The header of moof + mdat is set as the next auxiliary data.

Next, as illustrated in (c) of FIG. 30, the transmission device 15 uses the sample # 1, the sample # 2, and the sample # 3 among the samples # 1 to # 4 constituting the MPU to obtain the MP4. Among them, the moof + mdat header is used as the next auxiliary data.

Next, as shown in FIG. 30 (d), the transmission device 15 generates all the MP4s of the samples # 1 to # 4 constituting the MPU, of which the header of moof + mdat has movie fragment metadata. It becomes.

In addition, although the transmission apparatus 15 produced | generated auxiliary data for every sample here, you may produce | generate auxiliary data for every N samples. The value of N is an arbitrary number. For example, when transmitting auxiliary data M times when transmitting one MPU, N = all samples / M may be set.

Note that the information indicating the offset of the sample in the moof may be an offset value after a sample entry area having the number of subsequent samples is secured as a NULL area.

In addition, auxiliary data may be generated so as to have a configuration for fragmenting MF metadata.

[Reception operation example using auxiliary data]
Reception of auxiliary data generated as described with reference to FIG. 30 will be described. FIG. 31 is a diagram for explaining reception of auxiliary data. In FIG. 31A, the number of samples constituting the MPU is 30, and auxiliary data is generated and transmitted every 10 samples.

In FIG. 30A, sample # 1- # 10 is included in auxiliary data # 1, sample # 1- # 20 is included in auxiliary data # 2, and sample # 1- # 30 is included in MF metadata. Each piece of information is included.

Note that sample # 1- # 10, sample # 11- # 20, and sample # 21- # 30 are stored in one MMT payload, but may be stored in units of samples or NALs, or fragments Or may be stored in an aggregated unit.

The receiving device 20 receives MPU meta, sample, MF meta, and auxiliary data packets, respectively.

The receiving device 20 concatenates the sample data in the order of reception (backward), and after receiving the latest auxiliary data, updates the previous auxiliary data. In addition, the receiving device 20 can configure a complete MPU by finally replacing the auxiliary data with MF metadata.

When receiving the auxiliary data # 1, the receiving device 20 connects the data as shown in the upper part of (b) of FIG. 31 to configure MP4. As a result, the receiving apparatus 20 can parse the samples # 1- # 10 using the information of the MPU metadata and the auxiliary data # 1, and information on the PTS, DTS, offset, and size included in the auxiliary data Can be decoded based on

Further, when receiving the auxiliary data # 2, the receiving device 20 connects the data as shown in the middle part of FIG. 31B, and configures the MP4. As a result, the receiving apparatus 20 can parse the samples # 1 to # 20 using the information of the MPU metadata and the auxiliary data # 2, and the information about the PTS, DTS, offset, and size included in the auxiliary data is included. Based on this, decoding can be performed.

Further, when receiving the MF metadata, the receiving device 20 connects the data as shown in the lower part of FIG. 31B to configure the MP4. Accordingly, the receiving apparatus 20 can parse the samples # 1- # 30 using the MPU metadata and the MF metadata, and is based on the PTS, DTS, offset, and size information included in the MF metadata. Can be decrypted.

When there is no auxiliary data, the receiving device 20 can acquire sample information for the first time after receiving the MF metadata. Therefore, it is necessary to start decoding after receiving the MF metadata. However, since the transmission device 15 generates and transmits auxiliary data, the reception device 20 can acquire sample information using the auxiliary data without waiting for reception of the MF metadata, so that the decoding start time is advanced. be able to. Furthermore, when the transmission device 15 generates auxiliary data based on the moof described with reference to FIG. 30, the reception device 20 can parse using the conventional MP4 parser as it is.

Also, newly generated auxiliary data and MF metadata include sample information that overlaps with auxiliary data transmitted in the past. Therefore, even when past auxiliary data cannot be acquired due to packet loss or the like, MP4 is reconfigured by using newly acquired auxiliary data or MF metadata, and sample information (PTS, DTS, size, And offset) can be obtained.

Note that auxiliary data does not necessarily include information on past sample data. For example, auxiliary data # 1 may correspond to sample data # 1- # 10, and auxiliary data # 2 may correspond to sample data # 11- # 20. For example, as shown in FIG. 31 (c), the transmission device 15 may sequentially transmit complete MF metadata as a data unit and a unit obtained by fragmenting the data unit as auxiliary data.

In addition, the transmission device 15 may repeatedly transmit auxiliary data or MF metadata repeatedly in order to prevent packet loss.

Note that the MMT packet and the MMT payload in which the auxiliary data is stored include the MPU sequence number and the asset ID as well as the MPU metadata, the MF metadata, and the sample data.

The reception operation using the auxiliary data as described above will be described with reference to the flowchart of FIG. FIG. 32 is a flowchart of a reception operation using auxiliary data.

First, the receiving device 20 receives the MMT packet and analyzes the packet header and the payload header (S501). Next, the receiving apparatus 20 analyzes whether the fragment type is auxiliary data or MF metadata (S502). If the fragment type is auxiliary data, the receiving apparatus 20 overwrites and updates past auxiliary data (S503). . At this time, when there is no past auxiliary data of the same MPU, the receiving apparatus 20 uses the received auxiliary data as new auxiliary data as it is. Then, the receiving device 20 acquires a sample based on the MPU metadata, auxiliary data, and sample data, and performs decoding (S507).

On the other hand, if the fragment type is MF metadata, the receiving device 20 overwrites the past auxiliary data with the MF metadata in step S505 (S505). The receiving apparatus 20 acquires a sample in the form of a complete MPU based on the MPU metadata, the MF metadata, and the sample data, and performs decoding (S506).

Although not shown in FIG. 32, in step S502, the receiving device 20 stores the data in a buffer when the fragment type is MPU metadata, and moves backward for each sample when the fragment type is sample data. Store the concatenated data in the buffer.

When the auxiliary data cannot be acquired due to packet loss, the receiving apparatus 20 can overwrite the sample with the latest auxiliary data or use the past auxiliary data to decode the sample.

Note that the auxiliary data transmission cycle and the number of transmissions may be predetermined values. Information on the transmission cycle and the number of times (count, countdown) may be transmitted together with the data. For example, the data unit header may store a transmission cycle, the number of transmissions, and a time stamp such as initial_cpb_removal_delay.

It is possible to follow the CPB buffer model by transmitting auxiliary data including information on the first sample of the MPU at least once before the initial_cpb_removal_delay. At this time, the MPU time stamp descriptor stores a value based on the picture timing SEI.

Note that the transmission method in the reception operation in which such auxiliary data is used is not limited to the MMT method, and can be applied to the case of streaming transmission of packets configured in the ISOBMFF file format such as MPEG-DASH.

[Transmission method when one MPU is composed of a plurality of movie fragments]
In the description of FIG. 19 and subsequent figures, one MPU is composed of one movie fragment, but here, a case where one MPU is composed of a plurality of movie fragments will be described. FIG. 33 is a diagram showing a configuration of an MPU composed of a plurality of movie fragments.

In FIG. 33, a sample (# 1- # 6) stored in one MPU is divided into two movie fragments and stored. The first movie fragment is generated based on samples # 1- # 3, and a corresponding moof box is generated. The second movie fragment is generated based on samples # 4- # 6, and a corresponding moof box is generated.

The headers of the moof box and mdat box in the first movie fragment are stored in the MMT payload and MMT packet as movie fragment metadata # 1. On the other hand, the headers of the moof box and mdat box in the second movie fragment are stored in the MMT payload and MMT packet as movie fragment metadata # 2. In FIG. 33, the MMT payload storing the movie fragment metadata is hatched.

Note that the number of samples constituting the MPU and the number of samples constituting the movie fragment are arbitrary. For example, two movie fragments may be configured such that the number of samples constituting the MPU is the number of samples in GOP units, and the number of samples in half of the GOP units is a movie fragment.

Although an example in which two movie fragments (moof box and mdat box) are included in one MPU is shown here, the number of movie fragments included in one MPU is not limited to two but may be three or more. Also, the samples stored in the movie fragment may be divided into an arbitrary number of samples instead of the equally divided number of samples.

In FIG. 33, the MPU metadata unit and the MF metadata unit are stored in the MMT payload as data units, respectively. However, the transmission apparatus 15 may store units such as ftyp, mmpu, moov, and moof as data units in the MMT payload in units of data units, or store them in the MMT payload in units of fragmented data units. Also good. Further, the transmission device 15 may store the data unit in the MMT payload in a unit of aggregation.

In FIG. 33, samples are stored in the MMT payload in units of samples. However, the transmission apparatus 15 may configure a data unit not in units of samples but in units of NAL units or in units of a plurality of NAL units, and store them in the MMT payload in units of data units. Further, the transmission device 15 may store the data unit in the MMT payload in the unit of fragmenting, or may store the data unit in the MMT payload in the unit of aggregation.

In FIG. 33, MPUs are configured in the order of moof # 1, mdat # 1, moof # 2, and mdat # 2, and an offset is given to moof # 1 assuming that the corresponding mdat # 1 is behind. Yes. However, offset may be given assuming that mdat # 1 is ahead of moof # 1. In this case, however, movie fragment metadata cannot be generated in the form of moof + mdat, and the moof and mdat headers are transmitted separately.

Next, the transmission order of MMT packets when the MPU having the configuration described in FIG. 33 is transmitted will be described. FIG. 34 is a diagram for explaining the transmission order of MMT packets.

34 (a) shows a transmission order in the case of transmitting MMT packets in the MPU configuration order shown in FIG. Specifically, FIG. 34A shows MPU meta, MF meta # 1, media data # 1 (sample # 1- # 3), MF meta # 2, media data # 2 (sample # 4- # 6). ) Shows an example of transmission in the order.

In FIG. 34B, MPU meta, media data # 1 (sample # 1- # 3), MF meta # 1, media data # 2 (sample # 4- # 6), and MF meta # 2 are transmitted in this order. An example is shown.

(C) of FIG. 34 is transmitted in the order of media data # 1 (sample # 1- # 3), MPU meta, MF meta # 1, media data # 2 (sample # 4- # 6), and MF meta # 2. An example is shown.

MF meta # 1 is generated using samples # 1- # 3, and MF meta # 2 is generated using samples # 4- # 6. For this reason, when the transmission method of FIG. 34A is used, a delay due to encapsulation occurs in the transmission of the sample data.

On the other hand, when the transmission method of FIG. 34 (b) and FIG. 34 (c) is used, the sample can be transmitted without waiting for the generation of the MF meta. No end-to-end delay can be reduced.

Also, in (a) the transmission order of FIG. 34, one MPU is divided into a plurality of movie fragments, and the number of samples stored in the MF meta is smaller than in the case of FIG. The amount of delay due to encapsulation can be made smaller than in the case.

In addition to the method shown here, for example, the transmission device 15 may concatenate MF meta # 1 and MF meta # 2 and transmit them together at the end of the MPU. In this case, MF meta of different movie fragments may be aggregated and stored in one MMT payload. Also, MF meta of different MPUs may be aggregated and stored in the MMT payload.

[Reception method when one MPU is composed of a plurality of movie fragments]
Here, an operation example of the reception apparatus 20 that receives and decodes the MMT packets transmitted in the transmission order described in FIG. 34B will be described. FIG. 35 and FIG. 36 are diagrams for explaining such an operation example.

The receiving device 20 receives the MMT packet including the MPU meta, the sample, and the MF meta transmitted in the transmission order as shown in FIG. Sample data is concatenated in the order received.

The receiving apparatus 20 concatenates data to T1, which is the time when MF meta # 1 is received, as shown in (1) of FIG. 36, and configures MP4. As a result, the receiving apparatus 20 can acquire the samples # 1- # 3 based on the MPU metadata and the information on the MF meta # 1, and information on the PTS, DTS, offset, and size included in the MF meta. Can be decoded based on

Further, the receiving device 20 connects MP2 to T2, which is the time when MF meta # 2 is received, as shown in (2) of FIG. As a result, the receiving apparatus 20 can acquire the samples # 4- # 6 based on the MPU metadata and the information on the MF meta # 2, and based on the information on the PTS, DTS, offset, and size of the MF meta. Can be decrypted. Further, the receiving apparatus 20 concatenates the data as shown in (3) of FIG. 36, and configures MP4 to generate samples # 1- # 6 based on the information of MF meta # 1 and MF meta # 2. You may get it.

Since one MPU is divided into a plurality of movie fragments, the time required to acquire the first MF meta in the MPU is shortened, so that the decoding start time can be advanced. In addition, the buffer size for accumulating samples before decoding can be reduced.

Note that the transmission device 15 has a shorter time from the transmission (or reception) of the first sample in the movie fragment to the transmission (or reception) of the MF meta corresponding to the movie fragment than the initial_cpb_removal_delay specified by the encoder. The division unit of the movie fragment may be set so that By setting in this way, the reception buffer can follow the cpb buffer, and low-delay decoding can be realized. In this case, absolute time based on initial_cpb_removal_delay can be used for PTS and DTS.

Further, the transmission device 15 may divide movie fragments at equal intervals, or may divide subsequent movie fragments at shorter intervals than the previous movie fragment. As a result, the receiving apparatus 20 can always receive the MF meta including the information of the sample before decoding the sample, and can perform continuous decoding.

The following two methods can be used for calculating the absolute time of PTS and DTS.

(1) The absolute time of PTS and DTS is determined based on the reception time (T1 or T2) of MF meta # 1 and MF meta # 2 and the relative time of PTS and DTS included in MF meta.

(2) The absolute time of PTS and DTS is determined based on the absolute time signaled from the transmission side, such as the MPU time stamp descriptor, and the relative time of PTS and DTS included in the MF meta.

(2-A) The absolute time signaled by the transmission device 15 may be an absolute time calculated based on initial_cpb_removal_delay specified by the encoder.

(2-B) The absolute time signaled by the transmission device 15 may be an absolute time calculated based on the predicted value of the reception time of the MF meta.

Note that MF meta # 1 and MF meta # 2 may be repeatedly transmitted. By repeatedly transmitting the MF meta # 1 and the MF meta # 2, the receiving apparatus 20 can acquire the MF meta again even when the MF meta cannot be acquired due to packet loss or the like.

An identifier indicating the order of movie fragments can be stored in the payload header of the MFU including the samples constituting the movie fragment. On the other hand, an identifier indicating the order of MF metas constituting a movie fragment is not included in the MMT payload. For this reason, the receiving apparatus 20 identifies the order of the MF meta with packet_sequence_number. Alternatively, the transmission device 15 stores an identifier indicating what number of movie fragment the MF meta belongs to in control information (message, table, descriptor), MMT header, MMT payload header, or data unit header and performs signaling. May be.

The transmission device 15 transmits the MPU meta, the MF meta, and the sample in a predetermined transmission order that is determined in advance, and the reception device 20 performs reception processing based on the predetermined transmission order that is determined in advance. May be. The transmission device 15 may signal the transmission order, and the reception device 20 may select (determine) reception processing based on the signaling information.

The reception method as described above will be described with reference to FIG. FIG. 37 is a flowchart of the operation of the reception method described with reference to FIGS.

First, the receiving apparatus 20 determines (identifies) whether the data included in the payload is MPU metadata, MF metadata, or sample data (MFU) based on the fragment type indicated in the MMT payload (S601). , S602). If the data is sample data, the receiving apparatus 20 buffers the sample and waits for reception of MF metadata corresponding to the sample and start of decoding (S603).

On the other hand, if the data is MF metadata in step S602, the receiving apparatus 20 acquires sample information (PTS, DTS, position information, and size) from the MF metadata, and uses the acquired sample information. Based on the PTS and DTS, the sample is acquired based on the PTS and DTS, and presented (S604).

Although not shown, when the data is MPU metadata, the MPU metadata includes initialization information necessary for decoding. For this reason, the receiving device 20 accumulates this and uses it in step S604 for decoding the sample data.

When the receiving device 20 stores the received MPU data (MPU metadata, MF metadata, and sample data) in the storage device, the receiving device 20 rearranges the MPU configuration as described in FIG. 19 or FIG. Accumulate after.

Note that, on the transmission side, a packet sequence number is assigned to the MMT packet with respect to packets having the same packet ID. At this time, the packet sequence number may be given after the MMT packet including MPU metadata, MF metadata, and sample data is rearranged in the transmission order, or the packet sequence number is given in the order before rearrangement. Also good.

When the packet sequence numbers are given in the order before rearrangement, the receiving device 20 can rearrange the data in the MPU configuration order based on the packet sequence numbers, which facilitates storage.

[How to detect the beginning of an access unit and the beginning of a slice segment]
A method for detecting the head of the access unit and the head of the slice segment based on the information of the MMT packet header and the MMT payload header will be described.

Here, when storing non-VCL NAL units (access unit delimiters, VPS, SPS, PPS, SEI, etc.) together as data units in the MMT payload, and non-VCL NAL units as data units, Two examples are shown in the case where data are aggregated and stored in one MMT payload.

FIG. 38 is a diagram illustrating a case where non-VCL NAL units are individually set as data units and aggregated.

In the case of FIG. 38, the top of the access unit is the top data of the MMT payload including a data unit having an MMT packet with a fragment_type value of MFU, an aggregation_flag value of 1 and an offset value of 0. At this time, the Fragmentation_indicator value is 0.

In the case of FIG. 38, the top of the slice segment is an MMT packet having a fragment_type value of MFU, an aggregation_flag value of 0, and a fragmentation_indicator value of 00 or 01.

FIG. 39 is a diagram showing a case where non-VCL NAL units are collectively used as a data unit. The field value of the packet header is as shown in FIG. 17 (or FIG. 18).

In the case of FIG. 39, the top of the access unit is the top data of the payload in the packet whose Offset value is 0.

In the case of FIG. 39, the top of the slice segment has a value different from 0 in the Offset value, and the top data of the payload of the packet whose fragmentation indicator value is 00 or 01 is the top of the slice segment.

[Reception processing when packet loss occurs]
Normally, when transmitting MP4 format data in an environment where packet loss occurs, the receiving apparatus 20 restores the packet by ALFEC (Application Layer FEC), packet retransmission control, or the like.

However, if AL-FEC is not used in streaming such as broadcasting, packet loss cannot be restored.

The receiving device 20 needs to restart decoding of video and audio again after data is lost due to packet loss. For this purpose, the receiving device 20 needs to detect the head of the access unit or NAL unit and start decoding from the head of the access unit or NAL unit.

However, since the start code is not attached to the head of the MP4 format NAL unit, the receiving apparatus 20 cannot detect the head of the access unit or the NAL unit even if the stream is analyzed.

FIG. 40 is a flowchart of the operation of the receiving apparatus 20 when a packet loss occurs.

The receiving apparatus 20 detects a packet loss by using a packet sequence number, a packet counter, a fragment counter, or the like in the header of the MMT packet or MMT payload (S701), and determines which packet has been lost based on the preceding and following relationships (S702) .

When it is determined that no packet loss has occurred (No in S702), the receiving device 20 configures an MP4 file and decodes the access unit or the NAL unit (S703).

When it is determined that a packet loss has occurred (Yes in S702), the receiving apparatus 20 generates a NAL unit corresponding to the NAL unit that lost the packet from dummy data, and configures an MP4 file (S704). When receiving dummy data in the NAL unit, the receiving apparatus 20 indicates that the NAL unit type is dummy data.

Further, the receiving device 20 detects the head of the next access unit or NAL unit based on the method described in FIG. 17, FIG. 18, FIG. 38 and FIG. Decoding can be resumed (S705).

When a packet loss occurs, the receiving device 20 may resume decoding from the beginning of the access unit and the NAL unit based on information detected based on the packet header, or may be a dummy data NAL unit. Decoding may be resumed from the beginning of the access unit and the NAL unit based on the header information of the reconstructed MP4 file.

When receiving the MP4 file (MPU), the receiving device 20 may separately acquire and store (replace) packet data (NAL unit or the like) lost due to packet loss from broadcasting or communication.

At this time, when receiving the lost packet from communication, the receiving device 20 stores the information of the lost packet (packet ID, MPU sequence number, packet sequence number, IP data flow number, IP address, etc.) as a server. To obtain the packet. The receiving apparatus 20 may acquire not only lost packets but also packet groups before and after the lost packets.

[How to configure movie fragments]
Here, a method for configuring movie fragments will be described in detail.

33, the number of samples constituting a movie fragment and the number of movie fragments constituting one MPU are arbitrary. For example, the number of samples constituting a movie fragment and the number of movie fragments constituting one MPU may be a fixed number determined in a fixed manner or may be determined dynamically.

Here, by configuring the movie fragment so as to satisfy the following conditions on the transmission side (transmission device 15), it is possible to guarantee low-delay decoding in the reception device 20.

The conditions are as follows.

The transmitting device 15 receives the sample data so that the receiving device 20 can receive the MF meta including the information of the sample before the decoding time (DTS (i)) of the arbitrary sample (Sample (i)). An MF meta is generated using the divided units as movie fragments and transmitted.

Specifically, the transmission device 15 configures a movie fragment using samples (including the i-th sample) that have been encoded before DTS (i).

For example, the following method is used as a method of dynamically determining the number of samples constituting a movie fragment and the number of movie fragments constituting one MPU so as to guarantee low-delay decoding.

(1) At the start of decoding, the decoding time DTS (0) of the sample Sample (0) at the beginning of the GOP is a time based on initial_cpb_removal_delay. The transmission apparatus configures the first movie fragment using the encoded sample at a time before DTS (0). Further, the transmission device 15 generates MF metadata corresponding to the first movie fragment and transmits it at a time before DTS (0).

(2) The transmission apparatus 15 configures the movie fragment so as to satisfy the above-described conditions even in the subsequent samples.

For example, when the first sample of the movie fragment is the kth sample, the MF meta of the movie fragment including the kth sample is transmitted by the decoding time DTS (k) of the kth sample. The transmission device 15 determines that the k-th time is when the encoding completion time of the l-th sample is before DTS (k) and the encoding completion time of the (l + 1) -th sample is after DTS (k). A movie fragment is constructed using the l-th sample from the samples.

Note that the transmission device 15 may configure a movie fragment using the kth sample to the sample less than the lth sample.

(3) After the encoding of the last sample of the MPU is completed, the transmission device 15 configures a movie fragment using the remaining samples, generates MF metadata corresponding to the movie fragment, and transmits it.

Note that the transmission device 15 may configure a movie fragment using a part of the samples that have been encoded, without configuring the movie fragment by using all the samples that have been encoded.

In the above, an example is shown in which the number of samples constituting a movie fragment and the number of movie fragments constituting one MPU are dynamically determined based on the above conditions so as to guarantee low-delay decoding. It was. However, the method for determining the number of samples and the number of movie fragments is not limited to such a method. For example, the number of movie fragments constituting one MPU may be fixed to a predetermined value, and the number of samples may be determined so as to satisfy the above condition. Alternatively, the number of movie fragments constituting one MPU and the time at which the movie fragment is divided (or the code amount of the movie fragment) may be fixed to a predetermined value, and the number of samples may be determined so as to satisfy the above condition.

Further, when the MPU is divided into a plurality of movie fragments, information indicating whether the MPU is divided into a plurality of movie fragments, the attribute of the divided movie fragment, or the attribute of the MF meta for the divided movie fragment May be sent.

Here, the attribute of the movie fragment is information indicating whether the movie fragment is the first movie fragment of the MPU, the last movie fragment of the MPU, or any other movie fragment.

The MF meta attribute corresponds to the MF meta corresponding to the first movie fragment of the MPU, the MF meta corresponding to the last movie fragment of the MPU, or other movie fragments. This is information indicating whether the MF meta is to be performed.

Note that the transmission device 15 may store and transmit the number of samples constituting a movie fragment and the number of movie fragments constituting one MPU as control information.

[Receiver operation]
An operation of the receiving device 20 based on the movie fragment configured as described above will be described.

The receiving apparatus 20 determines the absolute time of each of the PTS and DTS based on the absolute time signaled from the transmission side, such as the MPU time stamp descriptor, and the relative time of the PTS and DTS included in the MF meta.

When the MPU is divided based on information on whether the MPU is divided into a plurality of movie fragments, the receiving apparatus 20 performs the following processing based on the attribute of the divided movie fragments. .

(1) When the movie fragment is the first movie fragment of the MPU, the receiving device 20 sets the absolute time of the PTS of the first sample included in the MPU time stamp descriptor and the relative time of the PTS and DTS included in the MF meta. To generate the absolute time of PTS and DTS.

(2) When the movie fragment is not the first movie fragment of the MPU, the receiving device 20 uses the relative time of the PTS and DTS included in the MF meta without using the information of the MPU time stamp descriptor, and uses the PTS and DTS. Generate the absolute time of.

(3) When the movie fragment is the last movie fragment of the MPU, the receiving apparatus 20 resets the PTS and DTS calculation processing (relative time addition processing) after calculating the absolute times of the PTS and DTS of all the samples. To do. The reset process may be performed on the movie fragment at the top of the MPU.

The receiving device 20 may determine whether the movie fragment is divided as described below. Further, the receiving device 20 may acquire movie fragment attribute information as described below.

For example, the receiving device 20 may determine whether or not the image is divided based on an identifier movie_fragment_sequence_number field value indicating the order of movie fragments indicated in an MMTP (MMT Protocol) payload header.

Specifically, the receiving apparatus 20 has a case where the number of movie fragments included in one MPU is 1, the movie_fragment_sequence_number field value is 1, and the field value has a value of 2 or more. The MPU may be determined to be divided into a plurality of movie fragments.

In addition, the receiving device 20 has one MPU in which the number of movie fragments is 1, the movie_fragment_sequence_number field value is 0, and there is a value other than 0 for the MPU. May be determined to be divided into a plurality of movie fragments.

Similarly, the attribute information of the movie fragment may be determined based on movie_fragment_sequence_number.

Note that, without using movie_fragment_sequence_number, whether the movie fragment is divided or the attribute information of the movie fragment may be determined by counting the transmission of a movie fragment or MF meta included in one MPU.

With the configuration of the transmission device 15 and the reception device 20 as described above, the reception device 20 can receive movie fragment metadata at intervals shorter than those of the MPU, and can start decoding with low delay. In addition, it is possible to perform decoding with low delay by using a decoding process based on the MP4 parsing method.

As described above, the reception operation when the MPU is divided into a plurality of movie fragments will be described using a flowchart. FIG. 41 is a flowchart of the receiving operation when the MPU is divided into a plurality of movie fragments. Note that this flowchart illustrates in more detail the operation of step S604 in FIG.

First, the receiving device 20 acquires MF metadata when the data type is MF meta based on the data type indicated in the MMTP payload header (S801).

Next, the receiving apparatus 20 determines whether the MPU is divided into a plurality of movie fragments (S802). If the MPU is divided into a plurality of movie fragments (Yes in S802), the received MF It is determined whether the metadata is the top metadata of the MPU (S803). When the received MF metadata is the MF metadata at the top of the MPU (Yes in S803), the receiving device 20 determines the absolute time of the PTS indicated in the MPU time stamp descriptor and the PTS indicated in the MF metadata. The absolute time of the PTS and DTS is calculated from the relative time of the DTS (S804), and it is determined whether it is the last metadata of the MPU (S805).

On the other hand, when the received MF metadata is not the MPU head MF metadata (No in S803), the receiving device 20 does not use the information of the MPU time stamp descriptor and the relative of the PTS and DTS indicated in the MF metadata. The absolute time of the PTS and DTS is calculated using the time (S808), and the process proceeds to step S805.

If it is determined in step S805 that it is the last MF metadata of the MPU (Yes in S805), the receiving device 20 resets the PTS and DTS calculation processing after calculating the absolute times of the PTS and DTS of all the samples. To do. If it is determined in step S805 that the MPU is not the last MF metadata (No in S805), the receiving apparatus 20 ends the process.

If it is determined in step S802 that the MPU is not divided into a plurality of movie fragments (No in S802), the receiving apparatus 20 acquires sample data based on MF metadata transmitted after the MPU. Then, PTS and DTS are determined (S807).

And although not shown in figure, the receiving apparatus 20 finally performs a decoding process and a presentation process based on the determined PTS and DTS.

[Problems that occur when dividing movie fragments and their solutions]
So far, a method for shortening the End-to-End delay by dividing the movie fragment has been described. Hereafter, a problem newly generated when a movie fragment is divided and its solution will be described.

First, a picture structure in encoded data will be described as a background. FIG. 42 is a diagram illustrating an example of a picture prediction structure in each TemporalId when temporal scalability is realized.

In coding schemes such as MPEG-4 AVC and HEVC (High Efficiency Video Coding), scalability in the time direction (temporal scalability) is realized by using B pictures (bidirectional reference prediction pictures) that can be referenced from other pictures. it can.

42 (T) in FIG. 42 is an identifier of a hierarchical structure of the coding structure, and TemporalId indicates a deeper hierarchy as the value increases. A square block represents a picture, Ix in the block is an I picture (intra-screen prediction picture), Px is a P picture (forward reference prediction picture), and Bx and bx are B pictures (bidirectional reference prediction pictures). Show. X of Ix / Px / Bx indicates a display order and represents the order in which pictures are displayed. An arrow between pictures indicates a reference relationship. For example, a picture of B4 indicates that a predicted image is generated using I0 and B8 as reference images. Here, it is prohibited for one picture to use, as a reference image, another picture having a TemporalId that is larger than its own TemporalId. Hierarchies are specified in order to provide temporal scalability. For example, when all pictures in FIG. 42 are decoded, a video of 120 fps (frame per second) can be obtained, but only a hierarchy of TemporalId of 0 to 3 is obtained. Is decoded to obtain 60 fps video.

FIG. 43 is a diagram showing the relationship between the decoding time (DTS) and the display time (PTS) in each picture of FIG. For example, the picture I0 shown in FIG. 43 is displayed after completion of decoding of B4 so that no gap occurs in decoding and display.

As shown in FIG. 43, when the prediction structure includes a B picture, the decoding order and the display order are different. Therefore, after the pictures are decoded in the receiving apparatus 20, the picture delay processing and the picture rearrangement are performed. (Reorder) processing is required.

The example of the picture prediction structure in the temporal scalability has been described above. Even when the temporal scalability is not used, depending on the prediction structure, a picture delay process and a reorder process may be required. is there. FIG. 44 is a diagram illustrating an example of a picture prediction structure that requires picture delay processing and reorder processing. Note that the numbers in FIG. 44 indicate the decoding order.

As shown in FIG. 44, depending on the prediction structure, the sample leading in the decoding order may be different from the sample leading in the presentation order. In FIG. 44, the sample leading in the presentation order is the decoding order. Becomes the fourth sample. FIG. 44 shows an example of the prediction structure, and the prediction structure is not limited to such a structure. In other prediction structures, the sample that is the head in the decoding order may be different from the sample that is the head in the presentation order.

FIG. 45 is a diagram showing an example in which an MPU configured in the MP4 format is divided into a plurality of movie fragments and stored in an MMTP payload and an MMTP packet, as in FIG. The number of samples constituting the MPU and the number of samples constituting the movie fragment are arbitrary. For example, two movie fragments may be configured such that the number of samples constituting the MPU is the number of samples in GOP units, and the number of samples in half of the GOP units is a movie fragment. One sample may be one movie fragment, or the samples constituting the MPU may not be divided.

FIG. 45 shows an example in which two movie fragments (moof box and mdat box) are included in one MPU, but the number of movie fragments included in one MPU may not be two. Three or more movie fragments may be included in one MPU, or the number of samples included in an MPU. Also, the samples stored in the movie fragment may be divided into an arbitrary number of samples instead of the equally divided number of samples.

The movie fragment metadata (MF metadata) includes information on the PTS, DTS, offset, and size of the sample included in the movie fragment, and the receiving device 20 can use the sample when decoding the sample. PTS and DTS are extracted from the MF meta including the above information, and decoding timing and presentation timing are determined.

Hereafter, for the sake of detailed explanation, the absolute value of the decoding time of i samples is described as DTS (i), and the absolute value of the presentation time is described as PTS (i).

The information of the i-th sample among the time stamp information stored in the moof in the MF meta is specifically the relative value of the decoding time of the i-th sample and the (i + 1) -th sample, and the i-th sample. Are the relative values of the decoding time and the presentation time of these samples, which are hereinafter referred to as DT (i) and CT (i).

Movie fragment metadata # 1 includes DT (i) and CT (i) of samples # 1- # 3, and movie fragment metadata # 2 includes DT (i of samples # 4- # 6. ) And CT (i).

Further, the PTS absolute value of the MPU head access unit is stored in an MPU time stamp descriptor or the like, and the receiving apparatus 20 calculates the PTS and DTS based on the PTS_MPU of the MPU head access unit and CT and DT. calculate.

FIG. 46 is a diagram for explaining a calculation method and problems of the PTS and DTS when the MPU is configured by the samples # 1 to # 10.

46A shows an example in which the MPU is not divided into movie fragments, and FIG. 46B shows an example in which the MPU is divided into two movie fragments in units of 5 samples. ) Shows an example in which the MPU is divided into 10 movie fragments in units of samples.

As described in FIG. 45, when the PTS and DTS are calculated using the MPU time stamp descriptor and the time stamp information (CT and DT) in the MP4, the first sample in the presentation order in FIG. Is the fourth in decoding order. Therefore, the PTS stored in the MPU time stamp descriptor is the PTS (absolute value) of the fourth sample in the decoding order. Hereinafter, this sample is referred to as A sample. Also, the first sample in the decoding order is called a B sample.

Since the absolute time information related to the time stamp is only the information of the MPU time stamp descriptor, the receiving apparatus 20 calculates the PTS (absolute time) and DTS (absolute time) of the other samples until the A sample arrives. Can not. The receiving device 20 cannot calculate the PTS and DTS of the B sample.

46A, the A sample is included in the same movie fragment as the B sample and stored in one MF meta. Therefore, the receiving apparatus 20 can determine the BTS DTS immediately after receiving the MF meta.

46B, the A sample is included in the same movie fragment as the B sample and stored in one MF meta. Therefore, the receiving apparatus 20 can determine the BTS DTS immediately after receiving the MF meta.

46 (c), the A sample is included in a movie fragment different from the B sample. For this reason, the receiving apparatus 20 can determine the DTS of the B sample only after receiving the MF meta including the CT and DT of the movie fragment including the A sample.

Therefore, in the case of the example of FIG. 46C, the receiving device 20 cannot start decoding immediately after the arrival of the B samples.

As described above, when the A sample is not included in the movie fragment including the B sample, the receiving apparatus 20 decodes the B sample unless the MF meta related to the movie fragment including the A sample is received. I can't start.

If the first sample in the presentation order and the first sample in the decoding order do not match, the movie fragment is divided before the A sample and the B sample are not stored in the same movie fragment. To do. In addition, this problem occurs regardless of whether the MF meta is post-feed or post-feed.

As described above, when the first sample in the presentation order does not match the first sample in the decoding order and the A sample and the B sample are not stored in the same movie fragment, immediately after the B sample is received, DTS cannot be determined. Therefore, the transmission device 15 separately transmits D sample (absolute value) of B samples or information that can calculate DTS (absolute value) of B samples on the receiving side. Such information may be transmitted using control information, a packet header, or the like.

The receiving device 20 calculates the DTS (absolute value) of the B sample using such information. FIG. 47 is a flowchart of the reception operation when the DTS is calculated using such information.

The receiving apparatus 20 receives the movie fragment at the head of the MPU (S901), and determines whether the A sample and the B sample are stored in the same movie fragment (S902). When stored in the same movie fragment (Yes in S902), the receiving apparatus 20 calculates the DTS using only the information of the MF meta without using the DTS (absolute time) of the B sample, and starts decoding. (S904). Note that in step S904, the reception apparatus 20 may determine the DTS using the BTS DTS.

On the other hand, if the A sample and the B sample are not stored in the same movie fragment in step S902 (No in S902), the receiving apparatus 20 acquires the DTS (absolute time) of the B sample, determines the DTS, and decodes it. Is started (S903).

In the above description, the absolute value of the decoding time and the absolute value of the presentation time of each sample are calculated using the MF meta (time stamp information stored in the MP4 format moof) in the MMT standard. Although the example has been described, it goes without saying that the MF meta may be implemented by replacing the absolute value of the decoding time and the absolute value of the presentation time of each sample with arbitrary control information that can be used for calculation. As an example of such control information, the relative value CT (i) of the decoding times of the i-th sample and the (i + 1) -th sample described above is used as the presentation time of the i-th sample and the (i + 1) -th sample. The control information replaced with the relative value, the relative value CT (i) of the decoding time of the i-th sample and the (i + 1) -th sample, and the relative value of the presentation time of the i-th sample and the (i + 1) -th sample There is control information including both.

(Embodiment 3)
[Overview]
In Embodiment 3, a content transmission method and data structure in the case of transmitting content such as video, audio, subtitles, and data broadcasting by broadcasting will be described. That is, a content transmission method and data structure specialized for broadcast stream reproduction will be described.

In the third embodiment, an example in which the MMT scheme (hereinafter also simply referred to as MMT) is used as the multiplexing scheme will be described, but other multiplexing schemes such as MPEG-DASH or RTP may be used. .

First, the details of the method of storing the data unit (DU: Data Unit) in the payload in MMT will be described. FIG. 48 is a diagram for explaining a method of storing data units in the payload in the MMT.

In MMT, the transmission apparatus stores a part of data constituting the MPU as a data unit in the MMTP payload, and transmits it with a header. The header includes an MMTP payload header and an MMTP packet header. The unit of the data unit may be a NAL unit unit or a sample unit.

48 (a) shows an example in which the transmission apparatus aggregates a plurality of data units and stores them in one payload. In the example of FIG. 48A, a data unit header (DUH: Data Unit Header) and a data unit length (DUL: Data Unit Length) are added to the head of each of the plurality of data units, and the data unit header and A plurality of data units to which the data unit length is assigned are collectively stored in the payload.

48 (b) shows an example in which one data unit is stored in one payload. In the example of FIG. 48B, a data unit header is added to the head of the data unit and stored in the payload. FIG. 48C shows an example in which one data unit is divided and a data unit header is added to the divided data unit and stored in the payload.

The data unit includes a timed-MFU that is a medium including information related to synchronization such as video, audio, or subtitles, a non-timed-MFU that is a medium not including information related to synchronization such as a file, MPU metadata, and MF metadata. The data unit header is determined according to the type of the data unit. Note that there is no data unit header in MPU metadata and MF metadata.

In addition, the transmission apparatus cannot, in principle, aggregate different types of data units, but may be defined so that different types of data units can be aggregated. For example, when the size of the MF metadata is small, such as when it is divided into movie fragments for each sample, the number of packets can be reduced by aggregating the MF metadata and the media data, and further the transmission capacity can be reduced. You can also.

When the data unit is MFU, some information of the MPU such as information for configuring the MPU (MP4) is stored as a header.

For example, the header of timed-MFU includes movie_fragment_sequence_number, sample_number, offset, priority, dependency_counter, etc., and the header of non-timed-MFU includes item_iD. The meaning of each field is shown in standards such as ISO / IEC 23008-1 or ARIB STD-B60. Hereinafter, the meaning of each field prescribed | regulated in such a standard is demonstrated.

Movie_fragment_sequence_number indicates the sequence number of the movie fragment to which the MFU belongs, and is also indicated in ISO / IEC 14496-12.

Sample_number indicates the sample number to which the MFU belongs, and is also indicated in ISO / IEC 14496-12.

Offset indicates the MFU offset amount in bytes in the sample to which the MFU belongs.

Priority indicates the relative importance of the MFU in the MPU to which the MFU belongs, and indicates that an MFU with a high priority number is more important than an MFU with a low priority number.

Dependency_counter indicates the number of MFUs whose decoding process depends on the MFU (that is, the number of MFUs that cannot be decoded unless this MFU is decoded). For example, when the MFU is HEVC and the B picture or P picture refers to an I picture, the B picture or P picture cannot be decoded unless the I picture is decoded.

Therefore, when the MFU is a sample unit, the dependency_counter in the MFU of the I picture indicates the number of pictures that refer to the I picture. When the MFU is in units of NAL units, the dependency_counter in the MFU belonging to the I picture indicates the number of NAL units belonging to the picture that refers to the I picture. Further, in the case of a video signal subjected to temporal direction hierarchical coding, the MFU of the enhancement layer depends on the MFU of the base layer. Therefore, the dependency_counter in the MFU of the base layer indicates the number of MFUs of the enhancement layer. This field can only be generated after the number of dependent MFUs has been determined.

Item_iD indicates an identifier that uniquely identifies the item.

[MP4 non-support mode]
As illustrated in FIG. 19 and FIG. 21, as a method for transmitting an MPU in the MMT, the transmission apparatus transmits MPU metadata or MF metadata before or after the media data, and only the media data. There is a way to send. In addition, the receiving apparatus includes a decoding method using a receiving apparatus and a receiving method compliant with MP4, and a decoding method without using a header.

As a data transmission method specialized for broadcast stream reproduction, for example, there is a transmission method that does not support MP4 reconfiguration in the receiving apparatus.

The transmission method that does not support MP4 reconfiguration in the receiving apparatus includes, for example, a method that does not transmit metadata (MPU metadata and MF metadata) as shown in FIG. In this case, the field value of the fragment type (information indicating the type of data unit) included in the MMTP packet is fixed to 2 (= MFU).

When metadata is not transmitted, as described above, an MP4 compliant receiving device or the like cannot decode received data as MP4, but can decode without using metadata (header). It is.

Therefore, metadata is not necessarily essential information for broadcast stream decoding and playback. Similarly, the data unit header information in timed-MFU described in FIG. 48 is information for reconfiguring MP4 in the receiving apparatus. Since it is not necessary to reconfigure MP4 in broadcast stream reproduction, the information of the data unit header in timed-MFU (hereinafter also referred to as timed-MFU header) is not necessarily information necessary for broadcast stream reproduction.

The receiving device can easily reconfigure MP4 by using metadata and information for reconfiguring MP4 in the data unit header (hereinafter also referred to as MP4 configuration information). However, even if only one of the metadata and the MP4 configuration information in the data unit header is transmitted, the receiving device cannot reconfigure MP4. There are few merits by transmitting only one of the metadata and the information for reconfiguring MP4, and generating and transmitting unnecessary information causes an increase in processing and a decrease in transmission efficiency.

Therefore, the transmission device controls the data structure and transmission of the MP4 configuration information using the following method. The transmission device determines whether to indicate MP4 configuration information in the data unit header based on whether the metadata is transmitted. Specifically, the transmission device indicates MP4 configuration information in the data unit header when metadata is transmitted, and does not indicate MP4 configuration information in the data unit header when metadata is not transmitted.

As a method for not indicating the MP4 configuration information in the data unit header, for example, the following method can be used.

1. The transmitting apparatus sets the MP4 configuration information as reserved and does not operate it. As a result, it is possible to reduce the processing amount on the transmission side that generates the MP4 configuration information (processing amount of the transmission device).

2. The transmission device deletes the MP4 configuration information and compresses the header. As a result, it is possible to reduce the processing amount on the transmission side for generating the MP4 configuration information, and it is possible to reduce the transmission capacity.

Note that, when the MP4 configuration information is deleted and the header is compressed, the transmission device may indicate a flag indicating that the MP4 configuration information has been deleted (compressed). The flag is indicated in a header (MMTP packet header, MMTP payload header, data unit header) or control information.

Further, information on whether or not metadata is transmitted may be determined in advance, or may be separately signaled to a header or control information and transmitted to the receiving device.

For example, information on whether the metadata corresponding to the MFU is transmitted may be stored in the MFU header.

On the other hand, the receiving apparatus can determine whether or not MP4 configuration information is indicated based on whether or not metadata is transmitted.

Here, when the data transmission order (for example, the order such as MPU metadata, MF metadata, and media data) is determined, the receiving apparatus determines whether the metadata has been received before the media data. May be determined.

When MP4 configuration information is indicated, the receiving apparatus can use the MP4 configuration information for MP4 reconfiguration. Alternatively, the receiving apparatus can use the MP4 configuration information for detecting the head of other access units or NAL units.

Note that the MP4 configuration information may be all or part of the timed-MFU header.

Also, the transmitting apparatus may determine whether to indicate the item id in the non-timed-MFU header based on whether the metadata is transmitted in the non-timed-MFU header as well.

The transmitting device may indicate the MP4 configuration information only in one of the timed-MFU and the non-timed-MFU. When MP4 configuration information is indicated only in one of them, the transmission device determines whether to indicate MP4 configuration information based on whether it is timed-MFU or non-timed-MFU in addition to whether metadata is transmitted. To do. The receiving apparatus can determine whether metadata is transmitted and whether MP4 configuration information is indicated based on a timed / non-timed flag.

In the above description, the transmission device determines whether to indicate MP4 configuration information based on whether metadata (both MPU metadata and MF metadata) is transmitted. However, the transmission apparatus may not indicate the MP4 configuration information when a part of the metadata (either MPU metadata or MF metadata) is not transmitted.

Also, the transmission apparatus may determine whether to indicate MP4 configuration information based on information other than metadata.

For example, a mode such as MP4 support mode / MP4 non-support mode is defined, and the transmission apparatus indicates MP4 configuration information in the data unit header in the case of MP4 support mode, and data in the case of MP4 non-support mode. The MP4 configuration information may not be indicated in the unit header. In the MP4 support mode, the transmission apparatus transmits metadata and indicates MP4 configuration information in the data unit header. In the MP4 non-support mode, the transmission apparatus transmits the metadata without transmitting metadata. May not indicate MP4 configuration information.

[Operation flow of transmitter]
Next, the operation flow of the transmission apparatus will be described. FIG. 49 is an operation flow of the transmission apparatus.

First, the transmitting device determines whether or not to transmit metadata (S1001). If it is determined that the metadata is to be transmitted (Yes in S1002), the transmission apparatus proceeds to Step S1003, generates MP4 configuration information, and transmits it in the header (S1003). In this case, the transmission device also generates and transmits metadata.

On the other hand, if it is determined that the metadata is not transmitted (No in S1002), the transmission device does not generate MP4 configuration information and transmits it without storing it in the header (S1004). In this case, the transmission device does not generate and transmit metadata.

Whether or not metadata is transmitted in step S1001 may be determined in advance, whether or not metadata is generated inside the transmission apparatus, and whether or not metadata is transmitted inside the transmission apparatus. It may be determined based on.

[Operation flow of receiver]
Next, the operation flow of the receiving apparatus will be described. FIG. 50 is an operation flow of the receiving apparatus.

First, the receiving apparatus determines whether metadata is transmitted (S1101). Whether the metadata is transmitted can be determined by monitoring the fragment type in the MMTP packet payload. Further, whether or not the data is transmitted may be determined in advance.

When it is determined that the metadata is transmitted (Yes in S1102), the receiving device reconfigures MP4 and executes a decoding process using the MP4 configuration information (S1103). On the other hand, when it is determined that the metadata is not transmitted (No in S1102), the MP4 reconfiguration process is not performed, and the decoding process is executed without using the MP4 configuration information (S1104).

The receiving apparatus can detect a random access point, detect an access unit head, detect a NAL unit head, and the like without using MP4 configuration information by using the method described so far. , Packet loss detection and return processing from packet loss can be performed.

For example, the head of the access unit is the head data of the MMT payload whose aggregation_flag value is 1. At this time, the Fragmentation_indicator value is 0.

Also, the head of the slice segment is the head data of the MMT payload having an aggregation_flag value of 0 and a fragmentation_indicator value of 00 or 01.

The receiving device can detect the head of the access unit and the slice segment based on the above information.

The receiving apparatus analyzes the NAL unit header in the packet including the head of the data unit whose fragmentation_indicator value is 00 or 01, and the type of the NAL unit is an AU delimiter, and the type of the NAL unit is a slice segment. May be detected.

[Broadcast simple mode]
So far, as a data transmission method specialized for broadcast stream reproduction, a method that does not support MP4 configuration information in the receiving apparatus has been described. However, a data transmission method specialized for broadcast stream reproduction is not limited to this. Absent.

For example, the following method may be used as a data transmission method specialized for broadcast stream reproduction.

• The transmission device does not have to use AL-FEC in a broadcast fixed reception environment. When AL-FEC is not used, FEC_type in the MMTP packet header is always fixed to 0.

· The transmission device may always use AL-FEC in the mobile reception environment of broadcasting and the communication UDP transmission mode. When AL-FEC is used, FEC_type in the MMTP packet header is always 0 or 1.

・ The sending device does not have to bulk transfer assets. When the asset is not bulk-transmitted, the location_information indicating the number of asset transmission locations in the MPT may be fixed to 1.

· The transmission device does not have to perform hybrid transmission of assets, programs, and messages.

In addition, for example, when the broadcast simple mode is defined and the broadcast apparatus is in the broadcast simple mode, the transmission apparatus is set to the MP4 non-support mode, or the data transmission method specialized for broadcast stream reproduction described above is used. It is good. Whether it is the broadcast simple mode may be determined in advance, or the transmission device may store a flag indicating the broadcast simple mode as control information and transmit it to the reception device.

Further, based on whether or not the MP4 non-support mode described in FIG. 49 (whether or not metadata is transmitted), the transmitting apparatus sets the broadcast simple mode as the broadcast simple mode. The data transmission method specialized for broadcast stream reproduction shown in FIG.

When the receiving apparatus is in the broadcast simple mode, the receiving apparatus can perform the decoding process without reconfiguring MP4, assuming that it is in the MP4 non-supporting mode.

Also, if the receiving device is in the broadcast simple mode, it can determine that the function is specialized for broadcasting, and can perform reception processing specialized for broadcasting.

As a result, when the broadcast simple mode is used, not only processing unnecessary for the transmission device and reception device can be reduced by using only the function specialized for broadcasting, but also unnecessary information is not compressed and transmitted. Thus, transmission overhead can be reduced.

Note that, when the MP4 non-support mode is used, hint information for supporting a storage method other than the MP4 configuration may be indicated.

Examples of storage methods other than the MP4 configuration include a method of directly storing MMT packets and IP packets, and a method of converting MMT packets into MPEG-2 TS packets.

In the MP4 non-support mode, a format that does not conform to the MP4 configuration may be used.

For example, in the MP4 non-support mode, the data stored in the MFU may not be in a format with a NAL unit size at the head of the NAL unit in MP4 format but with a byte start code. Good.

In MMT, the asset type indicating the asset type is described in 4CC registered in MP4REG (http://www.mp4ra.org), and when HEVC is used as a video signal, 'HEV1' or 'HVC1' is used. It is done. 'HVC1' is a format that may include a parameter set in the sample, and 'HEV1' is a format that does not include the parameter set in the sample and includes the parameter set in the sample entry in MPU metadata.

In the case of broadcast simple mode or MP4 non-support mode, if MPU metadata and MF metadata are not transmitted, it may be specified that a parameter set is always included in a sample. Further, it may be specified that the format of 'HVC1' is always taken regardless of whether 'HEV1' or 'HVC1' is shown in the asset type.

[Supplement 1: Transmitter]
As described above, when metadata is not transmitted, MP4 configuration information is reserved, and a transmitting apparatus that is not operated can be configured as shown in FIG. FIG. 51 is a diagram illustrating an example of a specific configuration of the transmission apparatus.

The transmission apparatus 300 includes an encoding unit 301, an adding unit 302, and a transmission unit 303. Each of the encoding unit 301, the adding unit 302, and the transmitting unit 303 is realized by, for example, a microcomputer, a processor, or a dedicated circuit.

The encoding unit 301 encodes a video signal or an audio signal to generate sample data. Specifically, the sample data is a data unit.

The assigning unit 302 assigns header information including MP4 configuration information to sample data that is data obtained by encoding a video signal or an audio signal. The MP4 configuration information is information for reconstructing the sample data as an MP4 format file on the receiving side, and the content differs depending on whether or not the presentation time of the sample data is determined.

As described above, the assigning unit 302 adds movie_fragment_sequence_number, sample_number, offset, priority to the header (header information) of timed-MFU, which is an example of sample data (sample data including information related to synchronization) whose presentation time is determined. And MP4 configuration information such as dependency_counter.

On the other hand, the adding unit 302 is an example of sample data (sample data that does not include information related to synchronization) for which the presentation time is not defined. Include.

Then, when the transmission unit 303 does not transmit metadata corresponding to the sample data (for example, in the case of (b) in FIG. 21), the adding unit 302 determines whether the presentation time of the sample data is determined. Accordingly, the header information not including the MP4 configuration information is added to the sample data.

Specifically, when the presentation time of the sample data is determined, the assigning unit 302 assigns the header information not including the first MP4 configuration information to the sample data, and the presentation time of the sample data is determined. If not, header information including second MP4 configuration information is added to the sample data.

For example, when the metadata corresponding to the sample data is not transmitted by the transmission unit 303 as illustrated in step S1004 in FIG. 49, the adding unit 302 sets the MP4 configuration information to “reserved” (fixed value). The MP4 configuration information is not substantially generated and is not substantially stored in the header (header information). The metadata includes MPU metadata and movie fragment metadata.

The transmission unit 303 transmits sample data to which header information is added. More specifically, the transmission unit 303 packetizes the sample data to which the header information is added by using the MMT method, and transmits the packetized data.

As described above, in the transmission method and the reception method specialized for the reproduction of the broadcast stream, it is not necessary to reconfigure the data unit into MP4 in the reception device. If the receiving device does not need to be reconfigured to MP4, the processing of the transmitting device is reduced by not generating unnecessary information such as MP4 configuration information.

On the other hand, the transmitting device must send necessary information, but it is necessary to maintain consistency with the standard so that unnecessary additional information need not be transmitted separately.

According to the configuration such as the transmission device 300, the MP4 configuration information is not transmitted, for example, by transmitting the necessary information based on the standard by setting the area in which the MP4 configuration information is stored to a fixed value. There is an effect that it is not necessary to transmit additional information. That is, the configuration of the transmission device and the processing amount of the transmission device can be reduced. Further, since unnecessary data is not transmitted, transmission efficiency can be improved.

[Supplement 2: Receiver]
Further, the receiving device corresponding to the transmitting device 300 may be configured as shown in FIG. 52, for example. FIG. 52 is a diagram illustrating another example of the configuration of the reception device.

The receiving device 400 includes a receiving unit 401 and a decoding unit 402. The receiving unit 401 and the decoding unit 402 are realized by, for example, a microcomputer, a processor, or a dedicated circuit.

The receiving unit 401 is sample data which is data obtained by encoding a video signal or an audio signal, and is provided with header information including MP4 configuration information for reconfiguring the sample data as an MP4 format file. Receive data.

The decoding unit 402 decodes the sample data without using the MP4 configuration information when the metadata corresponding to the sample data is not received by the receiving unit and the presentation time of the sample data is determined. To do.

For example, the decoding unit 402 executes the decoding process without using the MP4 configuration information when the metadata corresponding to the sample data is not received by the receiving unit 401 as shown in step S1104 of FIG.

Thereby, the configuration of the receiving apparatus 400 and the processing amount in the receiving apparatus 400 can be reduced.

(Embodiment 4)
[Overview]
In the fourth embodiment, a method for storing non-timed media that does not include information related to synchronization, such as a file, into an MPU and a method for transmission using an MMTP packet will be described. In the fourth embodiment, the MPU in the MMT is described as an example, but the same MP4 base DASH can be applied.

First, details of a method for storing non-timed media (hereinafter also referred to as “asynchronous media data”) in the MPU will be described with reference to FIG. FIG. 53 is a diagram illustrating a method for storing non-timed media in the MPU and a method for transmitting the MMTP packet.

The MPU storing non-timed media is composed of boxes such as ftyp, mmpu, moov, and meta, and stores information related to files stored in the MPU. A plurality of idat boxes can be stored in the meta box, and one file is stored as an item in the idat box.

Part of the ftyp, mmpu, moov, and meta boxes constitute one data unit as MPU metadata, and the item or idat box constitutes a data unit as MFU.

After the data unit is aggregated or fragmented, a data unit header, an MMTP payload header, and an MMTP packet header are added and transmitted as an MMTP packet.

FIG. 53 shows an example in which File # 1 and File # 2 are stored in one MPU. The MPU metadata is not divided and the MFU is divided and stored in the MMTP packet. However, the present invention is not limited to this, and may be aggregated or fragmented according to the size of the data unit. Also, MPU metadata may not be transmitted, in which case only MFU is transmitted.

The header information such as the data unit header indicates itemID (an identifier for uniquely identifying an item), and the MMTP payload header and MMTP packet header include a packet sequence number (sequence number for each packet) and an MPU sequence number. (MPU sequence number, unique number in the asset).

Note that the data structure of the MMTP payload header and the MMTP packet header other than the data unit header is the same as that of the timed media described above (hereinafter also referred to as “synchronous media data”), and includes aggregation_flag, fragmentation_indicator, fragment_counter, and the like. .

Next, a specific example of header information when a file (= Item = MFU) is divided and packetized will be described with reference to FIGS. 54 and 55. FIG.

54 and 55 are diagrams showing an example in which a plurality of pieces of divided data obtained by dividing a file are packetized and transmitted. Specifically, FIG. 54 and FIG. 55 show information (packet sequence number, fragment counter, fragmentation) included in any of the data unit header, MMTP payload header, and MMTP packet header, which is header information for each divided MMTP packet. Indicator, MPU sequence number, item ID). 54 is a diagram illustrating an example in which File # 1 is divided into M pieces (M <= 256), and FIG. 55 is an example in which File # 2 is divided into N pieces (256 <N). FIG.

The division data number indicates the index of the division data from the beginning of the file, and this information is not transmitted. That is, the divided data number is not included in the header information. The divided data number is a number given to a packet corresponding to each of a plurality of divided data obtained by dividing the file, and is a number given by adding 1 in ascending order from the first packet. is there.

The packet sequence number is a sequence number of packets having the same packet ID. In FIG. 54 and FIG. 55, the divided data at the beginning of the file is A, and consecutive numbers are given up to the last divided data of the file. The packet sequence number is a number given by adding 1 in ascending order from the divided data at the head of the file, and is a number corresponding to the divided data number.

The fragment counter indicates the number of a plurality of pieces of divided data after the divided data among a plurality of pieces of divided data obtained by dividing one file. The fragment counter indicates a remainder obtained by dividing the number of divided data by 256 when the number of divided data, which is the number of pieces of divided data obtained by dividing one file, exceeds 256. In the example of FIG. 54, since the number of divided data is 256 or less, the field value of the fragment counter is (M-divided data number). On the other hand, in the example of FIG. 55, since the number of divided data exceeds 256, the value obtained by dividing (N-divided data number) by 256 ((N-divided data number)% 256).

The fragmentation indicator indicates the state of division of data to be stored in the MMTP packet, whether it is the first divided data, the last divided data, or other divided data in the divided data unit, Or it is a value which shows whether it is one or more data units which are not divided | segmented. Specifically, the fragmentation indicator is “01” in the first divided data, “11” in the last divided data, “10” in the remaining divided data, and “10” in the undivided data unit. 00 ”.

In the present embodiment, when the number of divided data exceeds 256, it will be described as a remainder obtained by dividing the number of divided data by 256. However, the number of divided data is not limited to 256, and may be other numbers (predetermined numbers). May be.

54 and 55, when the file is divided and the conventional header information is added to each of a plurality of divided data obtained by dividing the file and transmitted, the receiving apparatus receives the received MMTP packet. There is no information for deriving the number of divided data in the original file (divided data number), the number of divided data of the file, or the divided data number and the number of divided data. For this reason, in the conventional transmission method, even if the MMTP packet is received, the divided data number and the number of divided data of the data stored in the received MMTP packet cannot be uniquely detected.

For example, as shown in FIG. 54, when the number of divided data is 256 or less and it is known that the number of divided data is 256 or less in advance, the fragment data number and the number of divided data are obtained by referring to the fragment counter. It is possible to specify. However, when the number of divided data is 256 or more, the divided data number and the number of divided data cannot be specified.

When the number of divided data of a file is limited to 256 or less, and the data size that can be transmitted in one packet is x [bytes], the maximum size of the file that can be transmitted is limited to x * 256 [bytes]. The For example, in broadcasting, x = 4k [bytes] is assumed, and in this case, the maximum size of a file that can be transmitted is limited to 4k * 256 = 1M [bytes]. Therefore, when it is desired to transmit a file exceeding 1 [Mbytes], the number of divided data of the file cannot be limited to 256 or less.

Also, for example, by referring to the fragmentation indicator, it is possible to detect the first divided data and the last divided data of the file, so the number of MMTP packets is counted until the MMTP packet including the last divided data of the file is received. Or after receiving the MMTP packet including the last divided data of the file, it is possible to calculate the divided data number and the number of divided data by combining with the packet sequence number. The division data number and the number of division data may be signaled by combining the numbers. However, if reception starts from an MMTP packet that includes divided data in the middle of the file (that is, divided data that is neither the first divided data nor the last divided data of the file), the divided data number or divided data of the divided data The number cannot be specified. The division data number and the number of division data of the division data can be specified only after receiving the MMTP packet including the division data at the end of the file.

54 and 55, the following method is used to uniquely determine the divided data number and the divided data number of the divided data of the file when a packet including the divided data of the file is received from the middle. Is used.

First, the division data number will be described.

For the divided data number, the packet sequence number in the first divided data of the file (item) is signaled.

Storing in control information for managing files as a signaling method. Specifically, in FIG. 54 and FIG. 55, the packet sequence number A of the divided data at the head of the file is stored in the control information. In the receiving apparatus, the value A is obtained from the control information, and the divided data number is calculated from the packet sequence number indicated in the packet header.

The divided data number of the divided data is obtained by subtracting the packet sequence number A of the first divided data from the packet sequence number of the divided data.

As the control information for managing the file, for example, there is an asset management table defined in ARIB STD-B60. In the asset management table, the file size, version information, and the like are indicated for each file, stored in a data transmission message, and transmitted. FIG. 56 is a diagram showing the loop syntax for each file in the asset management table.

If the area of the existing asset management table cannot be expanded, signaling may be performed using a partial 32-bit area of the item_info_byte field indicating item information. A flag indicating whether or not the packet sequence number in the first divided data of the file (item) is indicated in a partial area of item_info_byte may be indicated in, for example, the reserved_future_use field of the control information.

When a file such as a data carousel is repeatedly transmitted, a plurality of packet sequence numbers may be indicated, or the leading packet sequence number of a file to be transmitted immediately after may be indicated.

Not only the packet sequence number of the divided data at the beginning of the file, but any information that links the divided data number of the file and the packet sequence number may be used.

Next, the number of divided data will be described.

∙ The loop order for each file included in the asset management table may be defined as the file transmission order. As a result, the packet sequence numbers at the beginning of two files that are consecutive in the transmission order can be found, and therefore, by subtracting the first packet sequence number of the previously transmitted file from the first packet sequence number of the subsequently transmitted file, It is possible to specify the number of divided data of a file to be transmitted. That is, for example, if File # 1 shown in FIG. 54 and File # 2 shown in FIG. 55 are consecutive in this order, the last packet sequence number of File # 1 and the first packet sequence of File # 2 Numbers are assigned consecutive numbers.

Also, it may be specified so that the number of divided data of a file can be specified by specifying a file dividing method. For example, when the number of divided data is N, the size of each of the 1st to (N−1) th divided data is L, and the size of the Nth divided data is a fraction (item_size−L * (N−1)). By defining, the number of divided data can be calculated backward from item_size shown in the asset management table. In this case, an integer value obtained by rounding up (item_size / L) is the number of divided data. Note that the file dividing method is not limited to this.

Also, the number of divided data may be stored directly in the asset management table.

The receiving apparatus receives the control information by using the above method, and calculates the number of divided data based on the control information. Further, the packet sequence number corresponding to the divided data number of the file can be calculated based on the control information. If the timing of receiving the divided data packet is earlier than the timing of receiving the control information, the divided data number and the number of divided data may be calculated at the timing of receiving the control information.

Note that when the divided data number or the number of divided data is signaled using the above method, the divided data number or the number of divided data is not specified based on the fragment counter, and the fragment counter becomes unnecessary data. Therefore, in the transmission of asynchronous media, when the divided data number and information that can specify the number of divided data are signaled using the above method or the like, the fragment counter may not be used or the header may be compressed. . Thereby, the processing amount of the transmission device and the reception device can be reduced, and the transmission efficiency can be improved. That is, when transmitting asynchronous media, the fragment counter may be reserved (invalidated). Specifically, the value of the fragment counter may be a fixed value of “0”, for example. When receiving asynchronous media, the fragment counter may be ignored.

When storing synchronous media such as video and audio, the transmission order of the MMTP packets in the sending apparatus matches the arrival order of the MMTP packets in the receiving apparatus, and the packets are not retransmitted. In such a case, if it is not necessary to detect and reconfigure the packet loss, the fragment counter may not be operated. In other words, in this case, the fragment counter may be reserved (invalidated).

Even without using a fragment counter, it is possible to detect a random access point, to detect the head of an access unit, to detect the head of a NAL unit, and to perform decoding processing, packet loss detection, and return from packet loss. Can be processed.

Also, in the transmission of real-time content such as live broadcasting, transmission with lower delay is required, and it is required to sequentially packetize and transmit the data after encoding is completed. However, in real-time content transmission, the conventional fragment counter cannot determine the number of divided data when the first divided data is transmitted. Therefore, the transmission of the first divided data is completed when all the data units are encoded. After the number is determined, a delay occurs. Even in such a case, this delay can be reduced by not operating the fragment counter using the above method.

FIG. 57 is an operation flow for specifying the divided data numbers in the receiving apparatus.

The receiving device acquires control information describing file information (S1201). The receiving apparatus determines whether the packet sequence number at the beginning of the file is indicated in the control information (S1202). If the packet sequence number at the beginning of the file is indicated in the control information (Yes in S1202), the receiving device A packet sequence number corresponding to the divided data number of the divided data is calculated (S1203). Then, after receiving the MMTP packet in which the divided data is stored, the receiving apparatus specifies the divided data number of the file from the packet sequence number stored in the packet header of the acquired MMTP packet (S1204).

On the other hand, when the packet sequence number at the beginning of the file is not indicated in the control information (No in S1202), the receiving apparatus acquires the MMTP packet including the last divided data of the file and then acquires the packet header of the acquired MMTP packet. The segment data number is specified using the fragment indicator and the packet sequence number stored in (S1205).

FIG. 58 is an operation flow for specifying the number of divided data in the receiving apparatus.

The receiving apparatus acquires control information in which file information is described (S1301). The receiving apparatus determines whether the control information includes information that can calculate the number of divided data of the file (S1302), and determines that the information that can calculate the number of divided data is included, (Yes in S1302), the number of divided data is calculated based on the information included in the control information (S1303). On the other hand, if the receiving apparatus determines that the number of divided data cannot be calculated (No in S1302), the receiving apparatus obtains the MMTP packet including the last divided data of the file and stores it in the packet header of the obtained MMTP packet. The number of divided data is specified using the fragment indicator and the packet sequence number (S1304).

FIG. 59 is an operation flow for determining whether or not to operate the fragment counter in the transmission apparatus.

First, the transmitting apparatus determines whether the medium to be transmitted (hereinafter also referred to as “media data”) is a synchronous medium or an asynchronous medium (S1401).

If the result of determination in step S1401 is synchronous media (synchronous media in S1402), does the transmitting apparatus match the MMTP packet order of transmission / reception in an environment where synchronous media is transmitted, and does packet reconfiguration unnecessary when packet loss occurs? It is determined whether or not (S1403). If it is determined that the transmission apparatus is unnecessary (Yes in S1403), the transmission apparatus does not operate the fragment counter (S1404). On the other hand, when the transmitting apparatus determines that it is not necessary (No in S1403), it operates the fragment counter (S1405).

If the result of the determination in step S1401 is asynchronous media (asynchronous media in S1402), the transmitting device uses the method described above to determine whether the fragmented data number and the number of divided data are signaled. Decide whether to use the counter. Specifically, when the divided data number and the number of divided data are signaled (Yes in S1406), the transmitting apparatus does not operate the fragment counter (S1404). On the other hand, when the divided data number and the number of divided data are not signaled (No in S1406), the transmitting apparatus operates the fragment counter (S1405).

Note that, when the fragment counter is not operated, the transmission device may set the value of the fragment counter to “reserved” or perform header compression.

Note that the transmission apparatus may determine whether to signal the above-described divided data number and the number of divided data based on whether to operate the fragment counter.

In addition, when the synchronous media does not operate the fragment counter, the transmission device may signal the divided data number and the number of divided data using the method described above in the asynchronous media. Conversely, the operation of the synchronous media may be determined based on whether the asynchronous media operates the fragment counter. In this case, whether the fragment is operated can be the same operation in the synchronous media and the asynchronous media.

Next, a method for specifying the number of divided data and the divided data number (when using a fragment counter) will be described. FIG. 60 is a diagram for explaining a method for specifying the number of divided data and a divided data number (when a fragment counter is used).

As described with reference to FIG. 54, when the number of divided data is 256 or less and it is known in advance that the number of divided data is 256 or less, by referring to the fragment counter, the divided data number and the divided data are referred to. It is possible to specify the number of data.

When the number of divided data of a file is limited to 256 or less, when the data size that can be transmitted in one packet is x [bytes], the maximum size of a file that can be transmitted is limited to x * 256 [bytes]. For example, in broadcasting, x = 4k [bytes] is assumed, and in this case, the maximum size of a file that can be transmitted is limited to 4k * 256 = 1M [bytes].

When the file size exceeds the maximum size of a file that can be transmitted, the file is divided in advance so that the size of the divided file is equal to or less than x * 256 [bytes]. Each of the plurality of divided files obtained by dividing the file is treated as one file (item), further divided within 256, and divided data obtained by further dividing is divided into MMTP packets. Stored and transmitted.

Note that information indicating that the item is a divided file, the number of divided files, and the sequence number of the divided file may be stored in the control information and transmitted to the receiving device. Further, these pieces of information may be stored in the asset management table, or may be indicated by using a part of the existing field item_info_byte.

When the item is one divided file among a plurality of divided files obtained by dividing one file, the receiving device identifies another divided file and reconstructs the original file. Can do. Further, the receiving device can uniquely identify the number of divided data and the divided data number by using the number of divided files, the index of the divided file, and the fragment counter in the control information. Further, the number of divided data and the divided data number can be uniquely specified without using a packet sequence number or the like.

Here, it is desirable that each of the plurality of divided files obtained by dividing one file has the same item_id. When another item_id is assigned, the item_id of the first divided file may be indicated in order to uniquely refer to the file from other control information.

Also, a plurality of divided files may always belong to the same MPU. When a plurality of files are stored in the MPU, different types of files may not be stored, and a file obtained by dividing one file may be stored without fail. The receiving apparatus can detect the update of the file by confirming the version information for each MPU without confirming the version information for each item.

FIG. 61 is an operation flow of the transmission apparatus when the fragment counter is used.

First, the transmission device checks the size of the file to be transmitted (S1501). Next, the transmitting apparatus determines whether or not the file size exceeds x * 256 [bytes] (x is a data size that can be transmitted in one packet, for example, MTU size) (S1502), and the file size is x. * If it exceeds 256 [bytes] (Yes in S1502), the file is divided so that the size of the divided file is less than x * 256 [bytes] (S1503). Then, the divided file is transmitted as an item, and information on the divided file (for example, it is a divided file, a sequence number in the divided file, etc.) is stored in the control information and transmitted (S1504). On the other hand, when the file size is less than x * 256 [bytes] (No in S1502), the file is transmitted as an item as usual (S1505).

FIG. 62 is an operation flow of the receiving apparatus when the fragment counter is used.

First, the receiving apparatus acquires and analyzes control information related to file transmission such as an asset management table (S1601). Next, the receiving apparatus determines whether the desired item is a divided file (S1602). If it is determined that the desired file is a divided file (Yes in S1602), the receiving apparatus acquires information for reconstructing the file, such as the divided file and the index of the divided file, from the control information (S1603). . Then, the receiving apparatus acquires items constituting the divided file and reconstructs the original file (S1604). On the other hand, if the receiving apparatus determines that the desired file is not a divided file (No in S1602), the receiving apparatus acquires the file as usual (S1605).

In short, the transmitting apparatus signals the packet sequence number of the divided data at the head of the file. Further, the transmission apparatus signals information that can specify the number of divided data. Alternatively, the transmission apparatus defines a division rule that can specify the number of divided data. In addition, the transmission apparatus performs reserved or header compression without using the fragment counter.

When the packet sequence number of the data at the head of the file is signaled, the receiving device specifies the number of divided data and the number of the divided data from the packet sequence number of the data at the head of the file and the packet sequence number of the MMTP packet. .

From another viewpoint, the transmission device divides a file and divides data for each divided file and transmits the divided file. Signals information (sequence number, number of divisions, etc.) associated with a divided file.

The receiving device specifies the divided data number and the number of divided data by the fragment counter and the sequence number of the divided file.

This makes it possible to uniquely identify the divided data number and divided data. In addition, since the divided data number of the divided data can be specified when the divided data in the middle is received, the waiting time can be reduced and the memory can be reduced.

Also, by not operating the fragment counter, the configuration of the transmission / reception device can reduce the processing amount and improve the transmission efficiency.

FIG. 63 is a diagram showing a service configuration when the same program is transmitted by a plurality of IP data flows. Here, a part (video / audio) data of a program with service ID = 2 is transmitted by an IP data flow using the MMT method, and the same service ID and data different from the part of the data are advanced BS. An example is shown in which an IP data flow using a data transmission method (in this example, the file transmission protocol is different but the same protocol may be used) is transmitted.

The transmitting device multiplexes the IP data so that the receiving device can guarantee that the data composed of a plurality of IP data flows is ready by the decoding time.

The receiving apparatus can realize guaranteed receiver operation by processing based on the decoding time using data composed of a plurality of IP data flows.

[Supplement: Transmitter and receiver]
As described above, a transmission apparatus that transmits data without operating a fragment counter can be configured as shown in FIG. A receiving apparatus that receives data without operating a fragment counter can also be configured as shown in FIG. FIG. 64 is a diagram illustrating an example of a specific configuration of the transmission apparatus. FIG. 65 is a diagram illustrating an example of a specific configuration of the reception device.

The transmission apparatus 500 includes a dividing unit 501, a configuration unit 502, and a transmission unit 503. Each of the division unit 501, the configuration unit 502, and the transmission unit 503 is realized by, for example, a microcomputer, a processor, or a dedicated circuit.

The receiving device 600 includes a receiving unit 601, a determining unit 602, and a configuration unit 603. Each of the reception unit 601, the determination unit 602, and the configuration unit 603 is realized by, for example, a microcomputer, a processor, or a dedicated circuit.

Detailed description of each component of the transmission device 500 and the reception device 600 will be given in the description of the transmission method and the reception method, respectively.

First, the transmission method will be described with reference to FIG. FIG. 66 is an operation flow (transmission method) by the transmission apparatus.

First, the dividing unit 501 of the transmission device 500 divides data into a plurality of divided data (S1701).

Next, the configuration unit 502 of the transmission device 500 forms a plurality of packets by adding header information to each of the plurality of pieces of divided data to form a packet (S1702).

Then, the transmission unit 503 of the transmission device 500 transmits the plurality of configured packets (S1703). The transmission unit 503 transmits the divided data information and the invalid fragment counter value. The divided data information is information for specifying a divided data number and the number of divided data. Further, the divided data number is a number indicating the number of divided data among the plurality of divided data. The number of divided data is the number of divided data.

Thereby, the processing amount of the transmission apparatus 500 can be reduced.

Next, the reception method will be described with reference to FIG. FIG. 67 is an operation flow (reception method) by the reception apparatus.

First, the receiving unit 601 of the receiving device 600 receives a plurality of packets (S1801).

Next, the determination unit 602 of the reception device 600 determines whether or not the divided data information has been acquired from a plurality of received packets (S1802).

When the determination unit 602 determines that the divided data information has been acquired (Yes in S1802), the configuration unit 603 of the reception device 600 receives the fragment counter value included in the header information without using the value. Data is constructed from the plurality of packets thus obtained (S1803).

On the other hand, when the determination unit 602 determines that the divided data information has not been acquired (No in S1802), the configuration unit 603 uses the fragment counter value included in the header information to receive the received plurality of pieces of information. Data may be configured from the packet (S1804).

Thereby, the processing amount of the receiving device 600 can be reduced.

(Embodiment 5)
[Overview]
In the fifth embodiment, a transmission method of a transmission packet (TLV packet) when a NAL unit is stored in a multiplexing layer in the NAL size format will be described.

As explained in Embodiment 1, H. H.264 and H.264. When storing 265 NAL units in the multiplexing layer, there are the following two types. One is a so-called byte stream format in which a start code consisting of a specific bit string is added immediately before the NAL unit header. The other is a format called a NAL size format in which a field indicating the size of the NAL unit is added. The byte stream format is used in the MPEG-2 system, RTP, and the like, and the NAL size format is used in MP4, or DASH or MMT that uses MP4.

In the byte stream format, the start code is composed of 3 bytes, and an arbitrary byte (a byte whose value is 0) can be added.

On the other hand, in the NAL size format in general MP4, the size information is indicated by one of 1 byte, 2 bytes, and 4 bytes. This size information is indicated in the lengthSizeMinusOne field in the HEVC sample entry. When the value of the field is “0”, it indicates 1 byte, when it is “1”, it indicates 2 bytes, and when it is “3”, it indicates 4 bytes.

Here, in the ARIB STD-B60 “Media transport method by MMT in digital broadcasting” standardized in July 2014, the output of the HEVC encoder is a byte stream when storing the NAL unit in the multiplexing layer. In this case, the byte start code is removed, and the size of the NAL unit in bytes indicated by 32 bits (unsigned integer) is added immediately before the NAL unit as length information. The MPU metadata including the HEVC sample entry is not transmitted, and the size information is fixed to 32 bits (4 bytes).

In addition, in ARIB STD-B60 “Media transport system using MMT in digital broadcasting”, a buffer before video signal decoding is used in the reception buffer model that the transmission device considers when transmitting to guarantee the buffer operation in the reception device. Is defined as CPB.

However, there are the following problems. CPB in the MPEG-2 system and HRD in HEVC are defined on the assumption that the video signal is in a byte stream format. For this reason, for example, when rate control of a transmission packet is performed on the assumption that it is a byte stream format with a 3-byte start code, a transmission packet in a NAL size format with a 4-byte size area added is received The receiving device may not be able to satisfy the reception buffer model in ARIB STD-B60. Further, since the specific buffer size and the extraction rate are not shown in the reception buffer model in ARIB STD-B60, it is difficult to guarantee the buffer operation in the reception apparatus.

Therefore, in order to solve the above problems, the reception buffer model for guaranteeing the buffer operation in the receiver is defined as follows.

FIG. 68 shows a reception buffer model based on the reception buffer model defined in ARIB STD-B60, particularly when only the broadcast transmission path is used.

The reception buffer model includes a TLV packet buffer (first buffer), an IP packet buffer (second buffer), an MMTP buffer (third buffer), and a pre-decoding buffer (fourth buffer). In the broadcast transmission path, a dejitter buffer and an FEC buffer are not necessary and are omitted.

The TLV packet buffer receives a TLV packet (transmission packet) from the broadcast transmission path, and stores a variable length packet header (IP packet header, full header when IP packet is compressed, IP packet compression time) stored in the received TLV packet. IP packet obtained by converting an IP packet including a compressed header) and a variable-length payload into an IP packet (first packet) having a fixed-length IP packet header that is header-decompressed. Are output at a constant bit rate.

The IP packet buffer converts the IP packet into an MMTP packet (second packet) having a packet header and a variable-length payload, and outputs the MMTP packet obtained by the conversion at a constant bit rate. Note that the IP packet buffer may be merged with the MMTP buffer.

The MMTP buffer converts the output MMTP packet into a NAL unit, and outputs the NAL unit obtained by the conversion at a constant bit rate.

The pre-decoding buffer sequentially stores the output NAL units, generates an access unit from the plurality of stored NAL units, and outputs the generated access unit to the decoder at the timing of the decoding time corresponding to the access unit.

The reception buffer model shown in FIG. 68 is characterized in that the MMTP buffer and the pre-decoding buffer other than the TLV packet buffer and the IP packet buffer in the previous stage follow the reception buffer model in MPEG-2 TS.

For example, the MMTP buffer (MMTP B1) for video is composed of a transport buffer (TB) and a buffer corresponding to a multiplexing buffer (MB) in MPEG-2 TS. Further, the MMTP buffer (MMTP Bn) for audio is composed of a buffer corresponding to a transport buffer (TB) in MPEG-2 TS.

The buffer size of the transport buffer is the same as MPEG-2 TS and is a fixed value. For example, it is set to n times the MTU size (n may be a decimal or an integer, and is 1 or more).

Also, the MMTP packet size is specified so that the overhead rate of the MMTP packet header is smaller than the overhead rate of the PES packet header. As a result, the extraction rate RX1, RXn, RXs of the transport buffer in MPEG-2 TS can be applied as it is as the extraction rate from the transport buffer.

Also, the size of the multiplexing buffer and the extraction rate are the MB size and RBX1 in MPEG-2 TS, respectively.

In addition to the above receive buffer model, the following restrictions are set to solve the problem.

The HVC specification of HEVC assumes a byte stream format, and MMT is a NAL size format in which a 4-byte size area is added to the head of a NAL unit. Therefore, at the time of encoding, rate control is performed so as to satisfy HRD in the NAL size format.

In other words, the transmission device performs rate control of transmission packets based on the above-described reception buffer model and restrictions.

The receiving apparatus can perform a decoding operation that causes underflow or overflow by performing reception processing using the above-described signal.

Even if the head size area of the NAL unit is not 4 bytes, rate control is performed so as to satisfy the HRD in consideration of the head size area of the NAL unit.

Note that the extraction rate of the TLV packet buffer (the bit rate when the TLV packet buffer outputs an IP packet) is set in consideration of the transmission rate after the expansion of the IP header.

That is, after the TLV packet having a variable data size is input, the TLV header is removed and the IP header is decompressed (restored), and then the transmission rate of the output IP packet is considered. In other words, the increase / decrease amount of the header is considered with respect to the input transmission rate.

Specifically, the data size is variable length, the IP header compressed packet and the IP header uncompressed packet are mixed, and the IP header size varies depending on the packet type such as IPv4 and IPv6. The transmission rate of the output IP packet is not unique. For this reason, the average packet length of the variable length data size is determined, and the transmission rate of the IP packet output from the TLV packet is determined.

Here, in order to define the maximum transmission rate after expansion of the IP header, the transmission rate is determined assuming that the IP header is always compressed.

Also, when IPv4 and IPv6 packet types are mixed, or when packet types are specified without being distinguished, the transmission rate is determined assuming an IPv6 packet having a large header size and a large increase rate after header expansion. .

For example, when the average packet length of the TLV packet input to the TLV packet buffer is S, and all the IP packets stored in the TLV packet are IPv6 packets, and header compression is performed, The maximum output transmission rate after the expansion of the IP header is expressed by Equation (1).

More specifically, the average packet length S of the TLV packet is expressed by Equation (2).

And the IPv6 header compression amount is expressed by Equation (3).

IPv6 header compression amount = TLV header length−IPv6 header length−UDP header length = 3-40-8
In this case, the maximum output transmission rate after the removal of the TLV header and the expansion of the IP header is expressed by Equation (4).

FIG. 69 is a diagram illustrating an example in which a plurality of data units are aggregated and stored in one payload.

In the MMT method, when a data unit is aggregated, as shown in FIG. 69, a data unit length and a data unit header are added before the data unit.

However, for example, when a video signal in the NAL size format is stored as one data unit, as shown in FIG. 70, there are two fields indicating the size for one data unit, which overlap as information. . FIG. 70 is a diagram illustrating an example in which a plurality of data units are aggregated and stored in one payload, and a video signal in the NAL size format is used as one data unit. Specifically, the size of both the first size area in the NAL size format (hereinafter referred to as “size area”) and the data unit length field located in front of the data unit header in the MMTP payload header is the size. This field is duplicated as information. For example, when the length of the NAL unit is L bytes, L bytes are indicated in the size area, and L bytes + “length of the size area” (bytes) is indicated in the data unit length field. Although the values indicated by the size area and the data unit length field do not completely match, it can be said that they overlap because one value can be easily calculated from the other value.

In this way, when data including size information of data is stored as a data unit, and a plurality of the data units are aggregated and stored in one payload, the size information is duplicated, so the overhead is large, There is a problem that transmission efficiency is poor.

Therefore, in the transmission apparatus, when data including data size information is stored as a data unit and a plurality of data units are aggregated and stored in one payload, as shown in FIG. 71 and FIG. It can be stored.

71, it is conceivable that the NAL unit including the size area is stored as a data unit, and the data unit length conventionally included in the MMTP payload header is not indicated. FIG. 71 is a diagram showing the structure of the payload of an MMTP packet that does not indicate the data unit length.

Also, as shown in FIG. 72, a flag indicating whether the header is compressed (that is, indicating whether the data unit length is indicated) or information indicating the size of the size area may be newly stored in the header. Good. The location for storing the information indicating the length of the flag and the size area may be indicated in units of data units such as a data unit header, or may be indicated in units of aggregated data units (in units of packets). FIG. 72 is an example showing an example of storing information indicating a flag and the size of the size area in the extended area assigned to each packet. The storage location of the information shown above is not limited to this, and may be an MMTP payload header, an MMTP packet header, or control information.

On the receiving side, when the flag indicating whether or not the data unit length is compressed indicates that the data unit length is compressed, the length information of the size area inside the data unit is acquired and the size area is acquired. By acquiring the size area based on the length information, the data unit length can be calculated using the acquired size area length information and the size area.

By the above method, the amount of data can be reduced on the transmission side, and the transmission efficiency can be improved.

Note that instead of reducing the data unit length, overhead may be reduced by reducing the size area. When the size area is reduced, information indicating whether the size area is reduced or information indicating the length of the data unit length field may be stored.

Note that length information is also included in the MMTP payload header.

When storing a NAL unit including a size area as a data unit, the payload size area in the MMTP payload header may be reduced regardless of whether or not aggregation is performed.

Also, even when data that does not include a size area is stored as a data unit, if the data unit length is indicated by aggregation, the payload size area in the MMTP payload header may be reduced.

When the payload size area is reduced, a flag indicating whether or not the payload size area has been reduced, length information of the reduced size field, or length information of the size field that has not been reduced may be indicated.

FIG. 73 shows an operation flow of the receiving apparatus.

As described above, the sending device stores the NAL unit including the size area as a data unit, and the data unit length included in the MMTP payload header is not indicated in the MMTP packet.

Hereinafter, a case where the data unit length is indicated or not and a case where the size information of the size area is indicated in the MMTP packet will be described as an example.

The receiving apparatus determines whether the data unit includes a size area and the data unit length is reduced based on information transmitted from the transmission side (S1901).

If it is determined that the data unit length has been reduced (Yes in S1902), the length information of the size area inside the data unit is acquired, and then the size area inside the data unit is analyzed to calculate the data unit length. (S1903).

On the other hand, if it is determined that the data unit length has not been reduced (No in S1902), the data unit length is calculated from either the data unit length or the size area inside the data unit as usual (S1904).

Note that the flag indicating whether or not the data unit length has been reduced and the length information of the size area do not need to be transmitted if the receiving apparatus is known in advance. In this case, the receiving apparatus performs the process shown in FIG. 73 based on predetermined information.

[Supplement: Transmitter and receiver]
As described above, the transmission apparatus that performs rate control so as to satisfy the definition of the reception buffer model at the time of encoding can also be configured as shown in FIG. In addition, a receiving apparatus that receives and decodes a transmission packet transmitted from a transmitting apparatus can be configured as shown in FIG. FIG. 74 is a diagram illustrating an example of a specific configuration of the transmission apparatus. FIG. 75 is a diagram illustrating an example of a specific configuration of the reception device.

The transmission apparatus 700 includes a generation unit 701 and a transmission unit 702. Each of the generation unit 701 and the transmission unit 702 is realized by, for example, a microcomputer, a processor, or a dedicated circuit.

The receiving apparatus 800 includes a receiving unit 801, a first buffer 802, a second buffer 803, a third buffer 804, a fourth buffer 805, and a decoding unit 806. Each of the receiving unit 801, the first buffer 802, the second buffer 803, the third buffer 804, the fourth buffer 805, and the decoding unit (decoder) 806 is, for example, a microcomputer, a processor, a dedicated circuit, or the like It is realized by.

Detailed description of each component of the transmission device 700 and the reception device 800 will be given in the description of the transmission method and the reception method, respectively.

First, the transmission method will be described with reference to FIG. FIG. 76 is an operation flow (transmission method) by the transmission apparatus.

First, the generation unit 701 of the transmission apparatus 700 generates an encoded stream by performing rate control so as to satisfy a rule based on a predetermined reception buffer model in order to guarantee the buffer operation of the reception apparatus (S2001).

Next, the transmission unit 702 of the transmission apparatus 700 packetizes the generated encoded stream and transmits a transmission packet obtained by packetizing (S2002).

Note that the reception buffer model used in the transmission device 700 is a configuration including the first to fourth buffers 802 to 805 of the configuration of the reception device 800, and thus description thereof is omitted.

Thereby, the transmission device 700 can guarantee the buffer operation of the reception device 800 when data transmission is performed using a method such as MMT.

Next, the reception method will be described with reference to FIG. FIG. 77 is an operation flow (reception method) by the reception apparatus.

First, the receiving unit 801 of the receiving device 800 receives a transmission packet composed of a fixed-length packet header and a variable-length payload (S2101).

Next, the first buffer 802 of the receiving apparatus 800 converts a packet composed of a variable-length packet header and a variable-length payload stored in the received transmission packet into a fixed-length packet header whose header is expanded. The first packet obtained by the conversion is output at a constant bit rate (S2102).

Next, the second buffer 803 of the reception device 800 converts the first packet obtained by the conversion into a second packet configured by a packet header and a variable-length payload, and obtains the result by conversion. The received second packet is output at a constant bit rate (S2103).

Next, the third buffer 804 of the receiving apparatus 800 converts the output second packet into a NAL unit, and outputs the NAL unit obtained by the conversion at a constant bit rate (S2104).

Next, the fourth buffer 805 of the receiving apparatus 800 sequentially stores the output NAL units, generates an access unit from the plurality of stored NAL units, and decodes the generated access unit corresponding to the access unit. Is output to the decoder at the timing (S2105).

Then, the decoding unit 806 of the receiving device 800 decodes the access unit output by the fourth buffer (S2106).

Thereby, the receiving apparatus 800 can perform a decoding operation that causes underflow or overflow.

(Embodiment 6)
[Overview]
In the sixth embodiment, a transmission method and a reception method in the case where leap second adjustment is performed on time information which is a reference of a reference clock in the MMT / TLV transmission method will be described.

FIG. 78 is a diagram showing an MMT / TLV protocol stack defined in ARIB STD-B60.

In the MMT system, data such as video and audio is stored in a packet for each first data unit such as multiple MPUs (Media Presentation Units) and MFUs (Media Fragment Units), and an MMTP packet header is added. An MMTP packet as a predetermined packet is generated (converted into an MMTP packet). An MMTP packet as a predetermined packet is generated by adding an MMTP packet header to control information such as a control message in MMTP. The MMTP packet header includes a field for storing a 32-bit short format NTP (Network Time Protocol: stipulated in IETF RFC 5905), which can be used for QoS control of a communication line.

Also, the reference clock on the transmission side (transmitting device) is synchronized with the 64-bit long format NTP defined in RFC 5905, and based on the synchronized reference clock, PTS (Presentation Time Stamp) or DTS (Decode Time) A time stamp such as (Stamp) is given to the synchronization medium. Further, the reference clock information on the transmission side is transmitted to the reception side, and the reception device generates a system clock in the reception device based on the reference clock information received from the transmission side.

Specifically, the PTS and DTS are stored in the MPU time stamp descriptor and MPU extended time stamp descriptor, which are MMTP control information, stored in the MP table for each asset, and converted into MMTP packets as control messages. Then transmitted.

MMTP packetized data is given a UDP header or IP header and encapsulated in an IP packet. At this time, a set of packets having the same source IP address, destination IP address, source port number, destination port number, and protocol type in the IP header or UDP header is defined as an IP data flow. Since IP packets with the same IP data flow have redundant headers, some IP packets are header-compressed.

As the reference clock information, a 64-bit NTP timestamp is stored in the NTP packet and stored in the IP packet. At this time, in the IP packet storing the NTP packet, the source IP address, the destination IP address, the source port number, the destination port number, and the protocol type are fixed values, and the header of the IP packet is not compressed.

FIG. 79 is a diagram showing a configuration of a TLV packet.

As shown in FIG. 79, the TLV packet can include transmission control information such as an IP packet, a compressed IP packet, AMT (Address Map Table) and NIT (Network Information Table) as data. It is identified using an 8-bit data type. In the TLV packet, a 16-bit field is used to indicate the data length (in bytes), and then the data value is stored. The TLV packet has 1-byte header information before the data type, and the header information is stored in a header area of 4 bytes in total. The TLV packet is mapped to a transmission slot in the advanced BS transmission system, and mapping information is stored in TMCC (Transmission and Multiplexing Configuration Control) control information.

FIG. 80 is a diagram illustrating an example of a block diagram of the receiving device.

In the receiving apparatus, first, the transmission line encoded data is decoded and error-corrected by the demodulation means for the broadcast signal received by the tuner, and the TLV packet is extracted. The TLV / IP DEMUX means performs TLV DEMUX processing and IP DEMUX processing. The TLV DEMUX process performs a process according to the data type of the TLV packet. For example, when the TLV packet has a compressed IP packet, the compressed header of the compressed IP packet is restored. IP DEMUX performs processing such as header analysis of IP packets and UDP packets, and extracts MMTP packets and NTP packets.

The NTP clock generation means reproduces the NTP clock from the extracted NTP packet. MMTP DEMUX performs filtering processing of components such as video and audio and control information based on the packet ID stored in the extracted MMTP packet header. A time stamp descriptor stored in the MP table is acquired from the control information acquisition means, and a PTS / DTS for each access unit is calculated by the PTS / DTS calculation means. The time stamp descriptor includes both an MPU time stamp descriptor and an MPU extended time stamp descriptor.

The access unit playback means converts video and audio filtered from the MMTP packet into data to be presented. Specifically, the unit data to be presented is a video signal NAL unit, access unit, audio frame, subtitle presentation unit, or the like. The decoding presenting means decodes and presents the access unit at the time when the PTS / DTS of the access unit matches based on the reference time information of the NTP clock.

Note that the configuration of the receiving device is not limited to this.

Next, the time stamp descriptor will be described.

FIG. 81 is a diagram for explaining the time stamp descriptor.

The PTS and DTS are stored in the MPU time stamp descriptor as the first control information and the MPU extended time stamp descriptor as the second control information, which are the MMT control information, and stored in the MP table for each asset. After being converted into a MMTP packet as a message, it is transmitted.

FIG. 81 (a) is a diagram showing the structure of the MPU time stamp descriptor defined in ARIB STD-B60. The MPU time stamp descriptor includes, for each of a plurality of MPUs, the first (first) AU (hereinafter referred to as the first AU) of the plurality of access units (AU) as second data units stored in the MPU. , The presentation time information (first time information) indicating the PTS (absolute value indicated by 64-bit NTP) of “start AU”) is stored. That is, the presentation time information of the MPU given to the MPU is stored in the control information of the MMTP packet and transmitted.

FIG. 81 (b) shows the structure of the MPU extended time stamp descriptor. The MPU extended time stamp descriptor stores information for calculating the PTS and DTS of the AU included in each MPU of the plurality of MPUs. The MPU extended time stamp descriptor includes relative information (second time information) from the PTS of the first AU of the MPU stored in the MPU time stamp descriptor, and the PTS of each of the plurality of AUs included in the MPU. The DTS can be calculated based on both the MPU timestamp descriptor and the MPU extended timestamp descriptor. That is, the PTS and DTS of the AU other than the head AU included in the MPU are based on the PTS of the head AU stored in the MPU time stamp descriptor and the relative information stored in the MPU extended time stamp descriptor. Can be calculated.

In other words, the second time information is relative time information for calculating the PTS or DTS of each of the plurality of AUs with the first time information. That is, 2nd time information is information which shows PTS or DTS of each of several AU with 1st time information.

NTP is reference time information based on Coordinated Universal Time (UTC). The UTC performs leap second adjustment (hereinafter referred to as “leap second adjustment”) in order to adjust the difference from astronomical time based on the rotation speed of the earth. Specifically, the leap second adjustment is performed at 9:00 am Japan time, and is an adjustment for inserting or deleting one second.

FIG. 82 is a diagram for explaining leap second adjustment.

FIG. 82 (a) is a diagram showing an example of leap second insertion in Japan time. As shown in (a) of FIG. 82, in the leap second insertion, after Japan time 8:59:59, it becomes 8:59:59 at the time of 9:00: 00:59:59 Repeated twice.

FIG. 82 (b) shows an example of leap second deletion in Japan time. As shown in FIG. 82 (b), in the leap second deletion, after Japan time 8:59:58, it becomes 9: 00: 00: 00 at the time of 8:59:59, and 8:59:59 One second is deleted.

In the NTP packet, in addition to a 64-bit time stamp, a 2-bit lease_indicator is stored. The “leap_indicator” is a flag for notifying in advance that the leap second adjustment is performed, and indicates leap second insertion when the leap_indicator = 1, and indicates deletion of the leap second when the leap_indicator = 2. The advance notification includes a method of notifying from the beginning of the month in which the leap second adjustment is performed, a method of notifying 24 hours in advance, and a method of starting notification at an arbitrary time. Further, the lease_indicator becomes 0 at the time when the leap second adjustment is completed (9: 00: 00: 00). For example, when a prior notice is made 24 hours in advance, the time immediately before the leap second adjustment on the day when the leap second adjustment is performed from 9:00 the day before the leap second adjustment is performed in Japan time (that is, the leap second). In the case of insertion, the time of the first time is 8:59:59 units, and in the case of leap second deletion, the leap_indicator may be “1” or “2” by 8:59:58 times) Indicated.

Next, I will explain the issues when adjusting leap seconds.

FIG. 83 is a diagram showing the relationship among NTP time, MPU time stamp, and MPU presentation timing. The NTP time is a time indicated by NTP. The MPU time stamp is a time stamp indicating the PTS of the first AU in the MPU. The MPU presentation timing is a timing at which the receiving apparatus should present the MPU according to the MPU time stamp. Specifically, (a) to (c) of FIG. 83 respectively show an NTP time, an MPU time stamp, and a leap second when leap seconds are not adjusted, when leap seconds are inserted, and when leap seconds are deleted. It is a figure which shows the relationship of MPU presentation time.

Here, the case where the NTP time (reference clock) on the transmission side is synchronized with the NTP server, and the NTP time (system clock) on the reception side is synchronized with the NTP time on the transmission side will be described as an example. To do. In this case, the receiving apparatus performs reproduction based on the time stamp stored in the NTP packet transmitted from the transmission side. In this case, since both the NTP time on the transmission side and the NTP time on the reception side are synchronized with the NTP server, the adjustment of ± 1 second is performed when the leap second is adjusted. 83 is assumed to be common to the NTP time on the transmission side and the NTP time on the reception side. Note that description will be made assuming that there is no transmission delay.

83 indicates the time stamp of the first AU in the presentation order among the plurality of AUs included in each of the plurality of MPUs, and is generated (set) based on the NTP time indicated by the arrow. Is done. Specifically, the MPU presentation is performed by adding a predetermined time (for example, 1.5 seconds in FIG. 83) to the NTP time as the reference time information at the timing of generating the MPU presentation time information. Generate time information. The generated MPU time stamp is stored in the MPU time stamp descriptor.

The receiving device presents the MPU at the MPU presentation time based on the time stamp stored in the MPU time stamp descriptor.

In FIG. 83, it is assumed that the playback time of one MPU is 1 second, but the playback time of one MPU may be other playback time, for example, 0.5 seconds. 0.1 seconds may be sufficient.

83 (a), the receiving apparatus can present MPUs # 1- # 5 in order based on the time stamp stored in the MPU time stamp descriptor.

However, in (b) of FIG. 83, the presentation times of MPU # 2 and MPU # 3 overlap due to the leap second insertion. For this reason, if the receiving device presents an MPU based on the time stamp stored in the MPU time stamp descriptor, there are two MPUs to be presented in the same time zone of 9: 00: 00: 00. Therefore, it cannot be determined which of the two MPUs should be presented. In addition, since the MPU presentation time (8:59:59) indicated by the time stamp of MPU # 1 exists twice, the receiving apparatus should present the MPU presentation time among the two MPU presentation times. I can not judge.

In FIG. 83 (c), the MPU presentation time (8:59:59) indicated by the MPU time stamp of MPU # 3 does not exist at the NTP time due to the deletion of the leap second. 3 cannot be presented.

The receiving apparatus can solve the above-described problems when performing MPU decoding processing and presentation processing not based on time stamps. However, it is difficult for a receiving apparatus that performs processing based on a time stamp to perform different processing (processing that is not based on a time stamp) only when a leap second occurs.

Next, a method for solving the problem at the time of leap second adjustment by correcting the time stamp on the transmission side will be described.

FIG. 84 is a diagram for explaining a correction method for correcting the time stamp on the transmission side. Specifically, FIG. 84A shows an example of leap second insertion, and FIG. 84B shows an example of leap second deletion.

First, the case of leap second insertion will be described.

As shown in FIG. 84 (a), at the time of the leap second insertion, the time immediately before the leap second insertion (that is, the first 8:59:59 at the NTP time) is set as the A area, and after the leap second insertion. (That is, after the second time 8:59:59 at the NTP time) is set as the B region. The A region and the B region are temporal regions, which are time zones or periods. The MPU time stamp in (a) of FIG. 84 is the same as the time stamp described in FIG. 83, and is a time stamp generated (set) based on the NTP time at the timing of giving the MPU time stamp.

* In the case of leap second insertion, the time stamp correction method on the transmission side will be described specifically.

The transmission side (transmission device) performs the following processing.

1. The timing for giving the MPU time stamp (the timing indicated by the arrow in FIG. 84A) is included in the A area, and the MPU time stamp (the value of the MPU time stamp before correction) is 9:00:00. In the following cases, the MPU time stamp is reduced by 1 second, corrected by 1 second, and the corrected MPU time stamp is stored in the MPU time stamp descriptor. That is, when the MPU time stamp is generated based on the NTP time included in the A area and the MPU time stamp indicates 9:00: 00 or later, the MPU time stamp is corrected by −1 second. Here, “9:00: 00” is a time corresponding to Japan time as a reference time for the leap second adjustment (that is, a time calculated by adding 9 hours to UTC time). It is. Also, correction information, which is information indicating that correction has been made separately, is transmitted to the receiving apparatus.

2. When the timing for giving the MPU time stamp (the timing indicated by the arrow in FIG. 84A) is the B region, the MPU time stamp is not corrected. That is, when the MPU time stamp is generated based on the NTP time included in the area B, the MPU time stamp is not corrected.

In the receiving apparatus, the MPU is presented based on the MPU time stamp and the correction information indicating whether or not the MPU time stamp is corrected (that is, whether or not information indicating correction is included).

When the receiving apparatus determines that the MPU time stamp is not corrected (that is, when it is determined that the information indicating correction is not included), the time stamp stored in the MPU time stamp descriptor is the receiving apparatus. The MPU is presented at the time that coincides with the NTP time (including both before and after correction). That is, in the case of the MPU time stamp stored in the MPU time stamp descriptor of the MPU transmitted before the MPU whose MPU time stamp is corrected, the MPU time stamp is before the leap second insertion (that is, the first 8th time). : 59:59 units or earlier) present the MPU at a timing that coincides with the NTP time. When the MPU time stamp stored in the MPU time stamp descriptor of the received MPU is a time stamp transmitted after the corrected MPU time stamp, the leap second is inserted (that is, the second time 8:59: The MPU is presented at the timing that coincides with the NTP time of the 59th and subsequent cars.

In addition, when the MPU time stamp stored in the MPU time stamp descriptor of the received MPU is corrected, the receiving device inserts the leap second into the time stamp stored in the MPU time stamp descriptor ( That is, the MPU is presented based on the NTP time of the second time (8:59:59 and after).

Note that information indicating that the MPU time stamp value has been corrected is stored and transmitted in a control message, descriptor, table, MPU metadata, MF metadata, MMTP packet header, or the like.

Next, the case of leap second deletion will be described.

As shown in (b) of FIG. 84, when the leap second is deleted, the time immediately before the leap second deletion (that is, immediately before 9:00 in the NTP time) is set as the C region, and the time after the leap second deletion (that is, , 9:00 and after NTP time) is set as the D area. Note that the C region and the D region are temporal regions, which are time zones or periods. The MPU time stamp in (b) of FIG. 84 is the same as the time stamp described with reference to FIG. 83, and is a time stamp generated (set) based on the NTP time at the timing of giving the MPU time stamp.

* In the case of leap second deletion, the time stamp correction method on the transmission side will be described in detail.

The transmission side (transmission device) performs the following processing.

1. The timing for giving the MPU time stamp (the timing indicated by the arrow in FIG. 84B) is included in the C region, and the MPU time stamp (the value of the MPU time stamp before correction) is 8:59:59. In the following cases, the MPU time stamp is added by 1 second, corrected by 1 second, and the corrected MPU time stamp is stored in the MPU time stamp descriptor. That is, when the MPU time stamp is generated based on the NTP time included in the C region and the MPU time stamp indicates 8:59:59 or later, the MPU time stamp is corrected by +1 second. Note that “8:59:59” here is a time corresponding to Japan time as a reference time for the leap second adjustment (that is, a time calculated by adding 9 hours to the UTC time). This is the time obtained by subtracting -1 second from. Also, correction information, which is information indicating that correction has been made separately, is transmitted to the receiving apparatus. In this case, the correction information does not necessarily have to be transmitted.

2. When the timing for applying the MPU time stamp (the timing indicated by the arrow in FIG. 84B) is the D region, the MPU time stamp is not corrected. That is, when the MPU time stamp is generated based on the NTP time included in the D area, the MPU time stamp is not corrected.

The receiving device presents the MPU based on the MPU time stamp. If there is correction information indicating whether or not the MPU time stamp is corrected, the MPU may be presented based on the MPU time stamp and the correction information.

Through the above processing, even when the leap second adjustment is performed at the NTP time, it is possible to present a normal MPU using the MPU time stamp stored in the MPU time stamp descriptor in the receiving apparatus. .

Note that it is possible to signal whether the timing for giving the MPU time stamp is the A region, the B region, the C region, or the D region and notify the receiving side. That is, the MPU time stamp is generated based on the NTP time included in the A area, generated based on the NTP time included in the B area, or generated based on the NTP time included in the C area. Or whether it was generated based on the NTP time included in the D region, and may be notified to the receiving side. In other words, identification information indicating whether or not the MPU time stamp (presentation time) is generated based on the reference time information (NTP time) before the leap second adjustment may be transmitted. Since this identification information is given based on the leap_indicator included in the NTP packet, the leap second adjustment immediately before the leap second adjustment is performed from 9:00 the day before the leap second adjustment is performed in Japan time. It is set based on the time until the time (that is, when the leap second is inserted, the first time is 8:59:59, and when leap second is deleted, the time is 8:59:58). This is information indicating whether it is an MPU time stamp. In other words, was the identification information generated based on the NTP time from the time before a predetermined period (for example, 24 hours) to the time immediately before the time immediately before the leap second adjustment was performed for the MPU time stamp? This is information indicating whether or not.

Next, a method for solving the problem at the time of leap second adjustment by correcting the time stamp in the receiving apparatus will be described.

FIG. 85 is a diagram for explaining a correction method for correcting the time stamp in the receiving apparatus. Specifically, FIG. 85A shows an example of leap second insertion, and FIG. 85B shows an example of leap second deletion.

First, the case of leap second insertion will be described.

As shown in (a) of FIG. 85, when a leap second is inserted as in (a) of FIG. 84, the time immediately before the leap second is inserted (that is, the first 8:59:59 at the NTP time). Is the A region, and the time after the leap second insertion (that is, the second time after 8:59:59 at the NTP time) is the B region. The A region and the B region are temporal regions, which are time zones or periods. The MPU time stamp in (a) of FIG. 85 is the same as the time stamp described in FIG. 83, and is a time stamp generated (set) based on the NTP time at the timing of giving the MPU time stamp.

A specific description will be given of a method of correcting the time stamp in the receiving apparatus when the leap second is inserted.

The transmission side (transmission device) performs the following processing.

-Stores the MPU time stamp in the MPU time stamp descriptor and transmits it to the receiving device without correcting the generated MPU time stamp.

· Information indicating whether the timing to which the time stamp of the MPU is given is the A region or the B region is transmitted as identification information to the receiving apparatus. That is, identification information indicating whether the MPU time stamp is generated based on the NTP time included in the A area or based on the NTP time included in the B area is transmitted to the receiving apparatus.

The receiving device performs the following processing.

In the receiving apparatus, the MPU time stamp is corrected based on the MPU time stamp and the identification information indicating whether the timing at which the MPU time stamp is given is the A region or the B region.

Specifically, the following processing is performed.

1. The timing at which the MPU time stamp is given (the timing indicated by the arrow in FIG. 85A) is included in the A area, and the MPU time stamp (the MPU time stamp value before correction) is 9:00:00. In the following cases, the MPU time stamp is reduced by 1 second and corrected by 1 second. That is, when the MPU time stamp is generated based on the NTP time included in the A area and the MPU time stamp indicates 9:00: 00 or later, the MPU time stamp is corrected by −1 second. Here, “9:00: 00” is a time corresponding to Japan time as a reference time for the leap second adjustment (that is, a time calculated by adding 9 hours to UTC time). It is.

2. When the timing for applying the MPU time stamp (the timing indicated by the arrow in FIG. 85A) is the B region, the MPU time stamp is not corrected. That is, when the MPU time stamp is generated based on the NTP time included in the area B, the MPU time stamp is not corrected.

When the MPU time stamp is not corrected, the MPU is presented at the time when the MPU time stamp stored in the MPU time stamp descriptor matches the NTP time (including before and after correction) of the receiving apparatus.

That is, in the case of an MPU time stamp received before the MPU time stamp to be corrected, the MPU is presented at a time that matches the NTP time before the leap second insertion (that is, before the first 8:59:59 units). Also, in the case of an MPU time stamp received after the MPU time stamp to be corrected, the MPU is presented at a time that coincides with the NTP time after the leap second is inserted (second time 8:59:59 or later).

When correcting the MPU time stamp, the corrected MPU time stamp presents the MPU based on the NTP time after the leap second is inserted in the receiving device (that is, after the second 8:59:59 units).

Note that the transmission side has identification information indicating whether the timing at which the MPU time stamp is given is the A area or the B area is a control message, descriptor, table, MPU metadata, MF metadata, MMTP packet header. Store it in etc. and transmit it.

Next, the case of leap second deletion will be described.

As shown in (b) of FIG. 85, when the leap second is deleted, the time immediately before the leap second deletion (ie, immediately before 9:00 at the NTP time) is set to C as in (a) of FIG. The time after the leap second deletion (that is, after 9:00: 00: at the NTP time) is set as the D region. Note that the C region and the D region are temporal regions, which are time zones or periods. The MPU time stamp in (b) of FIG. 85 is the same as the time stamp described in FIG. 83, and is a time stamp generated (set) based on the NTP time at the timing of giving the MPU time stamp.

A specific description will be given of a method of correcting the time stamp in the receiving apparatus in the case of leap second deletion.

The transmission side (transmission device) performs the following processing.

· Information indicating whether the timing at which the time stamp of the MPU is given is the C region or the D region is transmitted to the receiving apparatus as identification information. That is, identification information indicating whether the MPU time stamp is generated based on the NTP time included in the C area or based on the NTP time included in the B area is transmitted to the receiving apparatus.

The receiving device performs the following processing.

In the receiving apparatus, the MPU time stamp is corrected based on the MPU time stamp and the identification information indicating whether the timing at which the MPU time stamp is given is the C region or the D region.

Specifically, the following processing is performed.

1. The timing for giving the MPU time stamp (the timing indicated by the arrow in FIG. 85B) is included in the C region, and the MPU time stamp (the value of the MPU time stamp before correction) is 8:59:59. In the following cases, the MPU time stamp value is added by 1 second and corrected by 1 second. That is, when the MPU time stamp is generated based on the NTP time included in the C region and the MPU time stamp indicates 8:59:59 or later, the MPU time stamp is corrected by +1 second. Note that “8:59:59” here is a time corresponding to Japan time as a reference time for the leap second adjustment (that is, a time calculated by adding 9 hours to the UTC time). This is the time obtained by subtracting -1 second from.

2. When the timing for giving the MPU time stamp (the timing indicated by the arrow in FIG. 85B) is the D region, the MPU time stamp is not corrected. That is, when the MPU time stamp is generated based on the NTP time included in the D area, the MPU time stamp is not corrected.

In the receiving apparatus, the MPU is presented based on the MPU time stamp and the corrected MPU time stamp.

Even in this case, similarly to the case of correcting the MPU time stamp on the transmission side described with reference to FIG. 84, the MPU time stamp application timing is the A region, the B region, or the C region. It may be signaled whether it is the D area or the D area and notified to the receiving side. The details of the notification are the same and will not be described.

It should be noted that the information indicating whether or not the MPU time stamp has been corrected and the timing at which the MPU time stamp has been added described in FIG. 84 and FIG. 85 is the A region, the B region, or the C region. Additional information (identification information) such as information indicating whether the area is the D area may be valid when the leap_indicator of the NTP packet indicates leap second deletion (leap_indicator = 2) or insertion (leap_indicator = 1). It may be effective from a predetermined time (for example, 3 seconds before leap second adjustment), or may be effective dynamically.

Also, the timing at which the validity period of the additional information ends or the timing at which the signaling of the additional information ends may be matched with the lap packet indicator_indicator, or any predetermined time (for example, 3 seconds before the leap second adjustment). May be effective from the beginning, or may be enabled dynamically.

Note that the 32-bit time stamp stored in the MMTP packet, the time stamp information stored in the TMCC, and the like are also generated and assigned based on the NTP time, and the same problem occurs. Therefore, in the case of a 32-bit time stamp or time stamp information stored in TMCC, the time stamp is corrected using the same method as in FIGS. 84 and 85, and processing based on the time stamp is performed in the receiving apparatus. can do. For example, when storing the additional information with respect to a 32-bit time stamp stored in the MMTP packet, it may be indicated by using an extension area of the MMTP packet header. In this case, additional information is indicated for the extension type of the multi-header type. Further, the time during which the “leap_indicator” is set may indicate additional information using a part of bits of a 32-bit or 64-bit time stamp.

In the example of FIG. 84, the corrected MPU time stamp is stored in the MPU time stamp descriptor, but the MPU time stamp before and after correction (that is, the uncorrected MPU time stamp and the corrected MPU time stamp). ) May be transmitted to the receiving device. For example, the MPU time stamp descriptor before correction and the MPU time stamp descriptor after correction may be stored in the same MPU time stamp descriptor, or may be stored in two MPU time stamp descriptors, respectively. Good. In this case, whether the MPU time stamp before correction or the MPU time stamp after correction is identified by the arrangement order of the two MPU time stamp descriptors, the description order in the MPU time stamp descriptor, etc. Also good. In addition, the MPU time stamp before correction is always stored in the MPU time stamp descriptor, and when correction is performed, the time stamp after correction may be stored in the MPU extended time stamp descriptor.

In this embodiment, the time in Japan time (9 am) is described as an example of the NTP time, but the time is not limited to Japan time. The leap second adjustment is based on UTC time and is corrected simultaneously worldwide. The Japan time is a time advanced by 9 hours with respect to the UTC time, and is represented by a value (+9) with respect to the UTC time. Thus, you may employ | adopt the time adjusted to different time by the time difference according to a place.

As described above, normal reception processing using a time stamp is possible by correcting the time stamp on the transmitting side or the receiving device based on the information indicating the timing at which the time stamp is given.

Note that a receiving device that continues decoding processing sufficiently before the leap second adjustment time may be able to perform decoding processing and presentation processing that does not use a time stamp, but the receiving device that has been selected immediately before the leap second adjustment time is In some cases, the time stamp cannot be determined and cannot be presented until after the leap second adjustment is completed. Even in that case, by using the correction method in the present embodiment, reception processing using a time stamp becomes possible, and tuning can be performed just before the leap second adjustment time.

FIG. 86 shows an operation flow on the transmission side (transmission device) and FIG. 87 shows an operation flow on the reception device when the MPU time stamp is corrected on the transmission side (transmission device) described in FIG.

First, the operation flow on the transmission side (transmission device) will be described with reference to FIG.

When a leap second is inserted or deleted, it is determined whether the timing of giving the MPU time stamp is the A area, the B area, the C area, or the D area (S2201). A case where no leap second is inserted or deleted is not shown.

Here, A area to D area are defined as follows.

Area A: Time until just before leap second insertion (first 8:59:59)
Area B: Time after leap second insertion (second time after 8:59:59)
C area: Time until leap second deletion (until 9:00: 00)
D area: Time after leap second deletion (after 9:00:00)
If it is determined in step S2201 that the area is A, and the MPU time stamp indicates 9:00:00 or later, the MPU time stamp is corrected by −1 second, and the corrected MPU time stamp is converted into the MPU time. Store in the stamp descriptor (S2202).

Then, the correction information indicating that the MPU time stamp is corrected is signaled and transmitted to the receiving apparatus (S2203).

If it is determined in step S2201 that the region is the C region and the MPU time stamp indicates 8:59:59 or later, the MPU time stamp is corrected by +1 second, and the corrected MPU time stamp is displayed in the MPU. Store in the time stamp descriptor (S2205).

If it is determined in step S2201 that the region is the B region or the D region, the MPU time stamp is stored in the MPU time stamp descriptor without correction (S2204).

Next, the operation flow of the receiving apparatus will be described with reference to FIG.

Based on the information signaled from the transmission side, it is determined whether or not the MPU time stamp is corrected (S2301).

When it is determined that the MPU time stamp is corrected (Yes in S2301), the MPU is presented based on the MPU time stamp at the NTP time after the leap second adjustment is performed in the receiving device (S2302).

When it is determined that the MPU time stamp is not corrected (No in S2301), the MPU is presented based on the MPU time stamp at the NTP time.

When correcting the MPU time stamp, the MPU corresponding to the MPU time stamp is presented in the section where the leap second is inserted.

On the contrary, when the MPU time stamp is not corrected, the MPU corresponding to the MPU time stamp is not presented in the section where the leap second is inserted, but is presented in the section where the leap second is not inserted.

FIG. 88 shows an operation flow on the transmission side and FIG. 89 shows an operation flow of the reception apparatus when the MPU time stamp is corrected in the reception apparatus described in FIG.

First, the operation flow on the transmission side (transmission apparatus) will be described with reference to FIG.

It is determined whether or not leap second adjustment (insertion or deletion) is performed based on the leap_indicator of the NTP packet (S2401).

If it is determined that the leap second adjustment is to be performed (Yes in S2401), the MPU time stamp assignment timing is determined, the identification information is signaled, and transmitted to the receiving device (S2402).

On the other hand, if it is determined that the leap second adjustment is not performed (No in S2041), the process ends with the normal operation.

Based on the identification information signaled by the transmission side (transmission apparatus), it is determined whether the timing of giving the MPU time stamp is the A region, the B region, the C region, or the D region. (S2501). Here, the A region to the D region are the same as those defined above, and thus description thereof is omitted. As in FIG. 87, the case where the leap second is not inserted or deleted is not shown.

If it is determined in step S2501 that the area is A and the MPU time stamp indicates after 9:00: 00, the MPU time stamp is corrected by −1 second (S2502).

If it is determined in step S2501 that the area is the C area and the MPU time stamp indicates 8:59:59 or later, the MPU time stamp is corrected by +1 second (S2504).

If it is determined in step S2501 that the region is the B region or the D region, the MPU time stamp is not corrected (S2503).

The receiving apparatus further presents the MPU based on the corrected MPU time stamp in a process not shown.

When correcting the MPU time stamp, the MPU corresponding to the MPU time stamp is presented based on the MPU time stamp at the NTP time after the leap second adjustment is performed.

In short, the transmission side (transmission device) determines the timing for giving the MPU time stamp corresponding to the MPU for each of the plurality of MPUs. As a result of the determination, if the timing is the time immediately before the leap second insertion and the MPU time stamp indicates 9:00 or later, the MPU time stamp is corrected by −1 second. Further, the correction information indicating that the MPU time stamp is corrected is signaled and transmitted to the receiving apparatus. If the MPU time stamp indicates 8:59:59 or later, the MPU time stamp is corrected by +1 second.

In addition, when the receiving apparatus indicates that the MPU time stamp is corrected based on whether the MPU time stamp indicated by the correction information signaled by the transmitting side (transmitting apparatus) is corrected, The MPU is presented based on the MPU time stamp at the NTP time after the leap second adjustment. When the MPU time stamp is not corrected, the MPU is presented based on the MPU time stamp at the NTP time before the leap second adjustment.

Further, the transmission side (transmission device) determines the timing for giving the MPU time stamp corresponding to the MPU for each of the plurality of MPUs, and signals the determined result. The receiving apparatus performs the following processing based on the information indicating the timing for adding the MPU time stamp signaled by the transmitting side. Specifically, when the information indicating the timing indicates the time immediately before the leap second insertion, and the MPU time stamp indicates 9:00: 00 or later, the MPU time stamp is set to − Correct for 1 second. Further, when the information indicating the timing is the time immediately before the leap second deletion and the MPU time stamp indicates 8:59:59 or later, the MPU time stamp is corrected by +1 second.

When the MPU time stamp is corrected, the receiving device presents the MPU based on the MPU time stamp at the NTP time after the leap second adjustment is performed. When the MPU time stamp is not corrected, the MPU is presented based on the MPU time stamp at the NTP time before the leap second adjustment.

In this way, by determining the timing of giving the MPU time stamp in the leap second adjustment, the MPU time stamp can be corrected, and the receiving apparatus can determine which MPU should be presented, and the MPU time stamp description. Appropriate reception processing using a child or MPU extended time stamp descriptor becomes possible. That is, even when the leap second adjustment is performed on the NTP time, the receiving apparatus can present a normal MPU using the MPU time stamp stored in the MPU time stamp descriptor.

[Supplement: Transmitter and receiver]
As described above, the transmission apparatus that stores the data constituting the encoded stream in the MPU and transmits the data can also be configured as shown in FIG. Also, a receiving apparatus that receives an MPU storing data constituting an encoded stream can be configured as shown in FIG. FIG. 90 is a diagram illustrating an example of a specific configuration of the transmission apparatus. FIG. 91 is a diagram illustrating an example of a specific configuration of the reception device.

The transmission apparatus 900 includes a generation unit 901 and a transmission unit 902. Each of the generation unit 901 and the transmission unit 902 is realized by, for example, a microcomputer, a processor, or a dedicated circuit.

The receiving apparatus 1000 includes a receiving unit 1001 and a reproducing unit 1002. Each of the receiving unit 1001 and the reproducing unit 1002 is realized by, for example, a microcomputer, a processor, or a dedicated circuit.

Detailed description of each component of the transmission device 900 and the reception device 1000 will be given in the description of the transmission method and the reception method, respectively.

First, the transmission method will be described with reference to FIG. FIG. 92 is an operation flow (transmission method) by the transmission apparatus.

First, the generation unit 901 of the transmission apparatus 900 generates presentation time information (MPU time stamp) indicating the presentation time of the MPU as the first data unit based on the NTP time as the reference time information received from the outside. (S2601).

Next, has the transmission unit 902 of the transmission device 900 generated based on the MPU, the presentation time information generated by the generation unit 901, and the NTP time before the leap second adjustment of the presentation time information (MPU time stamp)? The identification information indicating whether or not is transmitted (S2602).

As a result, the receiving device that has received the information transmitted from the transmitting device 900 can reproduce the MPU based on the identification information even when the leap second adjustment is performed, so that the MPU is reproduced at the intended time. it can.

Next, the reception method will be described with reference to FIG. FIG. 93 is an operation flow (reception method) by the reception apparatus.

First, the receiving unit 1001 of the receiving apparatus 1000 determines whether or not the MPU, the presentation time information indicating the MPU presentation time, and the presentation time information (MPU time stamp) are generated based on the NTP time before leap second adjustment. Is received (S2701).

Next, the reproducing unit 1002 of the receiving apparatus 1000 reproduces the MPU received by the receiving unit 1001 based on the presentation time information (MPU time stamp) and the identification information received by the receiving unit 1001 (S2702).

Thereby, the receiving apparatus 1000 can reproduce the MPU at the intended time even when the leap second adjustment is performed.

In the sixth embodiment, reproduction means processing including at least one of decoding and presentation.

(Modification of Embodiment 6)
Next, a specific example of a specific signaling method of additional information (identification information) described with reference to FIGS. 84 and 85 will be described.

Here, a description will be given of a signaling method using a combination of the MPU time stamp descriptor shown in FIG. 81 (a) and the MPU extended time stamp descriptor shown in FIG. 81 (b).

FIG. 94 is a diagram showing an example of extension of the MPU extension time stamp descriptor. The field indicated by the underline in FIG. 94 is a field that is newly added to the MPU extended time stamp descriptor shown in FIG. 81 (b).

NTP_leap_indicator in FIG. 94 indicates a flag indicating whether or not additional information (identification information) relating to NTP leap second adjustment is indicated in the MPU extended time stamp descriptor as the second control information. When this flag is set, mpu_presentation_time_type in the loop for each of the plurality of MPUs is valid.

The mpu_presentation_time_type in FIG. 94 indicates that the mpu_presentation_time of the same sequence number described in the MPU time stamp stamp descriptor is corrected when the MPU time stamp correction as described in FIG. 84 is performed on the transmitting side (transmitting apparatus). Show me how. Also, mpu_presentation_time_type is the A area when the mpu_presentation_time of the same sequence number described in the MPU timestamp stamp descriptor is given when the MPU timestamp correction as described in FIG. 85 is performed in the receiving apparatus. , B region, C region, or D region.

That is, the MPU extended time stamp descriptor stores identification information indicating whether or not the MPU time stamp is generated based on time information (NTP time) before leap second adjustment. Further, the MPU extended time stamp descriptor stores identification information corresponding to the MPU in each loop of the plurality of MPUs.

In step S2602, the transmission unit 902 of the transmission apparatus 900 includes the MPU and MPU time stamp descriptor as first control information in which the presentation time information (MPU time stamp) generated by the generation unit 901 is stored, An MPU extended time stamp descriptor as second control information in which identification information indicating whether or not the presentation time information is generated based on time information (NTP time) before the leap second adjustment is stored is transmitted.

In step S2701, the receiving unit 1001 of the receiving apparatus 1000 receives the MPU, the MPU time stamp descriptor, and the MPU extended time stamp descriptor. Then, the reproducing unit 1002 of the receiving apparatus 1000 reproduces the MPU received by the receiving unit 1001 based on the MPU time stamp descriptor and the MPU extended time stamp descriptor received by the receiving unit 1001. As described above, the receiving apparatus 1000 analyzes both the MPU time stamp descriptor and the MPU extended time stamp descriptor, thereby enabling a decoding process using the MPU time stamp at the time of leap second adjustment. Therefore, the receiving apparatus 1000 can reproduce the MPU at the intended time even when the leap second adjustment is performed. Also, since the identification information is stored in the MPU extended time stamp descriptor, the time information (NTP time) before the leap second adjustment of the MPU time stamp with the MPU time stamp descriptor corresponding to the existing standard (MPEG). ) Can be extended based on the function of signaling whether or not the As described above, since the conventional configuration can be used, even when the signaling function is expanded, the design change can be minimized, and the manufacturing cost of the transmission side (transmission device) and the reception device can be reduced. .

It should be noted that the timing for setting up the NTP_leap_indicator may be the same as that of the NTP packet in the NTP packet, or may be an arbitrary time while the leap_indicator is set up. That is, the NTP_leap_indicator is determined to be a value corresponding to the value of the leap_indicator in the NTP packet corresponding to the NTP time serving as the reference for generating the MPU time stamp.

(Modification 2 of Embodiment 6)
The problem related to the leap second adjustment described in the present embodiment is not limited to the case where the time based on the NTP time is used, but the problem occurs similarly even when the time based on the UTC time is used. is there.

When the time system is a time system based on UTC, and the presentation time information (MPU time stamp) is generated and assigned based on the time system, a method similar to the method described so far is used. Thus, even during leap second adjustment, it is possible to correct and process the presentation time information using the presentation time information and signaling information (additional information, identification information) on the receiving device side.

For example, in ARIB STD-B60, when an event message is transmitted for the purpose of notifying an application running on the receiver of the time (time designation information) for designating the operation from the broadcasting station, the event message descriptor Is stored in the MMTP packet, and the time at which the event message occurs is indicated in the event message descriptor. In this case, the data constituting the application is stored and transmitted in the MPU as the first data unit. There are a plurality of methods for specifying the time at which an event message occurs, and it is indicated by time_mode. In time_mode, when the time at which an event message occurs is expressed in UTC, the UTC time is given based on the time information given based on the time information before the leap second adjustment or based on the time information after the leap second adjustment. The identification information indicating whether the time information is present is signaled. That is, the identification information is information indicating whether or not the presentation time information is generated based on the time information before the leap second adjustment (NTP time), like the identification information described in the present embodiment. Such identification information (signaling information) is preferably indicated in the event message descriptor. For example, signaling can be performed using the reserved_future_use field of the event message descriptor.

That is, in step S2601, the generation unit 901 of the transmission device 900 generates time designation information indicating the operation time of the application based on the NTP time as the reference time information received from the outside. In step S2602, the transmission unit 902 of the transmission apparatus 900 includes the MPU, the time designation information generated by the generation unit 901, which indicates the operation time of the application (the time when the event message occurs), and the time information is An event message descriptor as control information in which identification information indicating whether or not the time information is before the leap second adjustment is transmitted.

Note that the transmission side (transmission device) determines a time zone or program that requires leap second adjustment, and prohibits use of time_mode expressed in UTC in a time zone or program that requires leap second adjustment. It may be specified that no time_mode is used.

Alternatively, on the transmitting side (transmitting apparatus), the UTC time at which the event message occurs is calculated, and it is determined whether the time at which the event message is generated is after the leap second adjustment. The UTC time after the second adjustment may be calculated and stored in the event message descriptor. Further, it may be separately signaled whether the presentation time information has been adjusted for leap seconds.

Similarly, the UTC (Universal Time Coordinated) -NPT (Normal Play Time) reference descriptor indicates the relationship between UTC and NPT using the UTC_Reference field and the NPT_Reference field. Whether the time information shown in the UTC_Reference field or the NPT_Reference field is the time information given based on the reference time information before the leap second adjustment (NTP time) or the time information given based on the reference time information after the leap second adjustment Is signaled. That is, the UTC-NPT reference descriptor includes the presentation time information generated by the generation unit 901 of the transmission device 900, and identification information indicating whether or not the presentation time information is time information before the leap second adjustment (NTP time). Is the control information that stores. The identification information can be signaled using a reserved field or the like in the UTC-NPT reference descriptor.

Also, the method for designating the presentation time of captions or superimposes described in ARIB STD-B60 or ARIB STD-B62 is indicated by TMD (Time Control Mode) in the additional identification information (Additional_Arib_Subtitle_Info). The additional identification information is stored in the asset loop of the MPT.

When the TMD (time control mode) is a mode that starts with an MPU time stamp or NPT (Normal Play Time), the presentation time of the caption or the character superimpose is referred to the MPU extended time stamp descriptor or UTC-NPT. Indicated in the descriptor. With the signaling information, it is possible to correct and process the time information using the time information and the signaling information on the receiving device side even when the leap second is adjusted.

When the time control mode is “0000” (UTC_TTML description), the time code in the ARIB-TTML subtitle indicates the presentation time corresponding to UTC. In this case, it is signaled whether the UTC time indicated by the time code is the time information given based on the time information before the leap second adjustment or the time information given based on the time information after the leap second adjustment. . The signaling information transmitted at this time is preferably shown in the ARIB-TTML caption.

When the time control mode is “0010” (UTC_TTML description), the time is based on the UTC time indicated in the reference_start_time field in the additional identification information. In this case, signaling is performed in the additional identification information in the same manner as when time control mode = “0000” (UTC_TTML description). In this case, on the transmission side (transmission device), the UTC time indicated in the reference_start_time field is a period (for example, a time period) that is determined in advance from the time immediately before the leap second adjustment is performed (for example, the time until 9:00: 00). 24 hours) Information indicating whether it is the time from the previous time (that is, 9:00 on the previous day when the leap second adjustment is performed) to the time immediately before is transmitted as identification information.

It should be noted that the transmitting side (transmitting device) determines a time zone or a program that requires leap second adjustment, prohibits the use of the time control mode expressed in UTC, and specifies that the time control mode not related to UTC is used. Also good.

The same applies to the MH-Expire descriptor.

(Embodiment 7)
A general broadcasting station facility has a plurality of transmission systems, and one system is used as a normal facility for an actual broadcasting service, and the other is operated as a redundant system facility for backup. Thereby, when an error occurs in the service provided to the viewer due to the cause of the transmission facility or the like, it is possible to switch to the redundant system facility. Here, in the MMT / TLV method, the following phenomenon occurs when switching the redundant system facility.

1. Adjustment of MPU sequence number Adjustment of the MPU sequence number is performed as pre-processing for switching the redundant equipment. As shown in FIG. 95, when the MPU sequence number is adjusted, there may be a discontinuity in the MPU sequence number before and after the adjustment. FIG. 95 shows an example in which MPU sequence numbers overlap.

The MPU time stamp for the access unit and the access unit is the same before and after the adjustment of the MPU sequence number, but the same MPU sequence number is assigned to different MPUs. The same MPU sequence number is given across.

Thereby, the MPU sequence number stored in the MMTP packet header overlaps with the MPU sequence number stored in the MPU time stamp descriptor and the MPU extended time stamp descriptor in different MPUs. Similarly, the MPU sequence number may be skipped or returned.

2. Packet sequence number discontinuity The packet sequence number and packet counter differ between normal equipment and redundant equipment, so when switching from normal equipment to redundant equipment, packet sequence numbers and packet counters are discontinuous. There is a case. As shown in FIG. 96, in normal operation, packet sequence numbers are continuous, but may be skipped, duplicated, and returned. FIG. 96 is a diagram for describing a case where packet sequence numbers are discontinuous at the timing of switching from normal equipment to redundant equipment.

When the MPU sequence number or the packet sequence number becomes discontinuous as in 1 or 2 above, there is a problem that a receiving apparatus that operates by believing in the continuity of these sequence numbers may malfunction.

Note that the continuity of the MPU time stamp for the access unit and the access unit is maintained even when the MPU sequence number or the packet sequence number is discontinuous, and continuous data is sent to the receiving device.

A description will be given of reception processing when a discontinuity in MPU sequence number or packet sequence number as described with reference to FIGS. 95 and 96 occurs.

When a discontinuity occurs in the MPU sequence number or the packet sequence number, a receiving apparatus that performs processing using the MPU sequence number or the packet sequence number as shown below may malfunction.

As a process using the MPU sequence number, for example, there is a process of detecting the MPU boundary by switching the MPU sequence number in the MMTP packet header.

In addition, as the processing using the MPU sequence number, for example, the MPU time stamp descriptor and the MPU extended time stamp descriptor describe the time stamp for the MPU sequence number, and decoding and / or using the time stamp. Present.

As the processing using the packet sequence number, for example, the packet sequence number is used for detecting the packet loss, and when the packet sequence number is skipped, it is determined that the packet is lost, and when the packet sequence number is returned, it is determined that the packet is replaced. There is processing.

In order to prevent malfunction caused by performing processing using these MPU sequence numbers and packet sequence numbers, the receiving apparatus determines the discontinuity of the MPU sequence number and the packet sequence number. Performs processing that does not use numbers or packet sequence numbers.

Here, for example, the following method can be used as a method for determining the discontinuity of the MPU sequence number and the packet sequence number.

First, one of the determination methods may be signaling additional information indicating switching to a redundant system. Thereby, the receiving apparatus can perform a receiving process that does not use the MPU sequence number or the packet sequence number based on the additional information.

As a method of signaling additional information, there is a method of storing and transmitting the additional information in a control message, descriptor, table, MPU metadata, MF metadata, MMTP packet header, extended header of MMTP packet, or the like.

Next, a method for detecting discontinuity using other information in the receiving apparatus will be described.

For example, as information indicating the MPU boundary, there are a method of detecting the switching of the MPU sequence number in the MMTP payload header and a method of using RAP_flag which is a flag indicating whether or not the MMTP packet header is a random access point. In addition, it is possible to detect the MPU boundary even when the packet is changed to 0 when the Sample_number is 0, or when the fragment type is changed. Also, the MPU boundary can be detected by counting the number of access units.

Here, when there is a possibility that switching of redundant equipment is performed, the MPU boundary using the switching of the MPU sequence number is not determined, and the MPU boundary is determined using another method. For example, a packet whose RAP_flag is 1 is determined as the head packet in the MPU. Here, when the MPU sequence number of the packet after the MPU boundary is not different from the MPU sequence number of the previous packet, it is determined that the duplication of the MPU sequence number has occurred in the MMTP packet.

Alternatively, the packet whose RAP_flag is 1 is determined as the head packet in the MPU. Here, when the MPU sequence number of the packet determined to be the head packet is skipped or returned from the MPU sequence number of the previous packet, it is determined that the discontinuity of the MPU sequence number has occurred in the MMTP packet.

If the MPU sequence number discontinuity in the MMTP payload header always occurs before the MPU sequence number or packet sequence number discontinuity in the MPU time stamp descriptor or MPU extended time stamp descriptor, it is based on the determination result. Thus, a reception process that does not use a time stamp or a process that does not use a packet sequence number may be performed.

In addition, the discontinuity of the MPU sequence number in the MMTP payload header of the video packet is different from the discontinuity of the MPU sequence number in the MMTP payload header of the audio packet, or the MPU sequence number in the audio MPU time stamp descriptor or MPU extended time stamp descriptor. If it always occurs before the discontinuity of the packet sequence number, based on the detection result of the discontinuity of the MPU sequence number of the video packet, the reception processing that does not use the audio time stamp or the processing that does not use the packet sequence number May be implemented. The determination result may be determined as the timing at which switching to the redundant facility starts.

Note that the receiving apparatus does not use the MPU sequence number or the packet sequence number only when additional information indicating switching to the redundant facility is indicated by using a method of signaling additional information indicating switching to the redundant facility. Processing may be performed.

Note that the reception apparatus detects the discontinuity of the RAP_flag and the MPU sequence number only when additional information indicating switching to redundant equipment is indicated using a method of signaling additional information indicating switching to redundant equipment. Processing may be performed.

Here, the determination using the time stamp descriptor in the receiving apparatus will be described. Normally, a time stamp for the MPU sequence number is written and transmitted multiple times, but when the time stamp value is updated even though the MPU sequence number is the same, the receiving apparatus Is determined to be duplicated.

In addition, when the relationship between the time stamp value and the MPU sequence number does not correspond, such as when the MPU sequence number is updated by 2 or more even though the time stamp value has been updated for the presentation time of one MPU, it is received. The apparatus determines that an MPU sequence number discontinuity has occurred.

Note that the version in the MP table may not be updated when the MPU time stamp or the MPU extended time stamp is updated. Alternatively, version information may be separately defined, and the version of the time stamp information and the version of information other than the time stamp information may be transmitted separately. Thereby, the version frequency of information other than time stamp information can be reduced, and reception processing can be reduced.

In addition, when the discontinuity of the MPU sequence number occurs, the relationship between the MPU time stamp and the time stamp information cannot be trusted. Therefore, it may be signaled to the descriptor that the time stamp information is untrustworthy information. For example, using the same field as NTP_leap_indicator and mpu_presentation_time_type described with reference to FIG. 94, the timing for giving the MPU time stamp may be the timing before and after switching the redundant system, and may indicate that the time stamp is not reliable. Further, it may be indicated in the MPU extended time stamp descriptor on the assumption that it is used in combination with the MPU time stamp descriptor.

By using the above method, it is possible to perform processing without using the MPU sequence number or the packet sequence number by determining that the MPU sequence number or the discontinuity of the packet sequence number occurs in the receiving apparatus, and malfunctions in the synchronized playback can be prevented. It becomes possible to prevent.

The operation flow of the receiving apparatus when the discontinuity of the MPU sequence number and the packet sequence number as described above occurs will be described. FIG. 97 is an operation flow by the receiving apparatus when a discontinuity of the MPU sequence number or the packet sequence number occurs.

The receiving device determines the MPU boundary from the packet header or payload header in the MMTP packet. Specifically, the MPU boundary is determined by determining that the packet with RAP_flag equal to 1 is the MPU head packet. Further, the MPU sequence number is determined, and it is determined whether or not the MPU sequence number of the previous packet is increased by 1 (S2801).

When it is determined that RAP_flag is 1 and the MPU sequence number is increased by 1 (Yes in S2801), it is assumed that the MPU sequence number is not discontinuous, and reception processing using the MPU sequence number is performed. (S2803).

On the other hand, if it is determined that RAP_flag is 0 or RAP_flag is 1 and the MPU sequence number of the previous packet has not increased by 1 (No in S2801), a discontinuity in MPU sequence number has occurred. It is assumed that there is an MPU sequence number, and reception processing is performed (S2802).

(Embodiment 8)
Although the sixth embodiment has mainly described the correction of the MPU time stamp, a method of correcting the PTS or DTS for each AU included in the MPU in the receiving apparatus will be described. That is, a correction method for correcting the time stamp indicating the PTS or DTS of each of a plurality of other AUs excluding the head AU among the plurality of AUs included in the MPU in the receiving apparatus will be described.

FIG. 98 is a diagram for explaining a correction method for correcting the time stamp in the receiving apparatus when the leap second is inserted. FIG. 98 is the same case as FIG.

Here, the NTP time on the transmission side is synchronized with the NTP server, and the NTP time on the reception side is reproduced based on the time stamp stored in the NTP packet transmitted from the transmission side, The case where the transmission side is synchronized with the NTP time will be described as an example. In this case, both the NTP time on the transmission side and the NTP time on the reception side are adjusted by +1 second when a leap second is inserted.

Suppose that the NTP time in FIG. 98 is common to the NTP time on the transmission side and the NTP time on the reception side. In the following description, it is assumed that there is no transmission delay. In the following, only the correction method of the AU PTS in the MPU will be described, but the AU DTS in the MPU can also be corrected by using the same method. In FIG. 98, it is assumed that the number of AUs included in one MPU is 5, but the number of AUs included in an MPU is not limited to this.

As in the transmission method described in FIG. 85A, the NTP time (up to the first 8:59:59 units) immediately before the leap second insertion is the A region, and the NTP time after the leap second insertion (second time). 8:59:59 or later) is set as the B region.

The transmission side (transmission device) performs the following processing.

As described in Embodiment 6 with reference to FIG. 83, an MPU time stamp that is MPU presentation time information (first time information) is generated.

-Stores the MPU time stamp in the MPU time stamp descriptor and transmits it to the receiving device without correcting the generated MPU time stamp. That is, the first time information is time information that is not corrected on the transmitting side of the MPU during leap second adjustment. Also, identification information indicating whether the timing at which the MPU time stamp is given is the A area or the B area is transmitted to the receiving apparatus. Specifically, an MMTP packet storing a plurality of MPUs is transmitted. The MMTP packet includes, as control information, an MPU time stamp descriptor including an MPU time stamp as first time information, an MPU extended time stamp descriptor including relative information as second time information, and identification information. .

In FIG. 98, the timing at which the MPU time stamp is added is indicated by an arrow.

The receiving device performs the following processing.

As described in the sixth embodiment, in the MPU extended time stamp descriptor, the second time information for calculating the PTS or DTS for each AU includes the first AU of the MPU stored in the MPU time stamp descriptor. Stored as relative information from the PTS. At this time, the relative information stored in the MPU extended time stamp descriptor is not a value considering the insertion of leap seconds.

First, the receiving apparatus receives the MMTP packet including the MPU, the first time information, the second time information, and the identification information transmitted from the transmitting side (transmitting apparatus). As shown in “PTS”, all AUs included in the MPU based on both the first time information stored in the MPU time stamp descriptor and the second time information stored in the MPU extended time stamp descriptor. Each PTS (DTS) is calculated.

Next, the receiving apparatus corrects the PTS for each AU. The correction target AU has the MPU time stamp assignment timing of the MPU to which the AU belongs (the timing indicated by the arrow in FIG. 98) in the A region, and the PTS for each AU before correction is 9:00: AU indicating 00 or later. The AUs to be corrected are AUs indicated by underline in FIG. 98, and are AU # 5 in MPU # 1 and AU # 1-AU # 5 in MPU # 2. If the receiving apparatus determines that the AU is a PTS correction target AU, the receiving apparatus corrects the PTS of the AU by −1 second.

That is, in the correction, for each of a plurality of AUs, the MPU time stamp of the MPU storing the AU is generated based on the reference time information before the leap second adjustment, and the calculated PTS of the AU or It is determined whether or not the DTS satisfies a correction condition after a predetermined time (9:00 in the case of leap second insertion). In the correction, the PTS or DTS of the AU determined to satisfy the above correction condition is corrected in the determination.

The AU to which the MPU time stamp of the MPU to which the AU belongs is in the A region and the PTS is not corrected (in FIG. 98, AU up to AU # 4 in MPU # 1) is before the leap second insertion (1 The AU is presented at the time that coincides with the NTP time of 8:59:59: before the second). Further, the AU with corrected PTS or the AU after that (in FIG. 98, the AU after AU # 5 in MPU # 1) is at the NTP time after the leap second is inserted (second time after 8:59:59 units). AU is presented at the matching time.

Note that the identification information indicating the timing of giving the MPU time stamp needs to be presented to at least the MPU including the AU to be corrected. That is, it is not always necessary to present identification information to an MPU that does not include a correction target AU.

FIG. 99 is a diagram for explaining a correction method for correcting the time stamp in the receiving apparatus when the leap second is deleted. FIG. 99 is the same case as FIG. 85 (b).

The NTP time on the transmission side is synchronized with the NTP server, and the NTP time on the reception side is reproduced based on the time stamp stored in the NTP packet transmitted from the transmission side, so that the transmission side NTP time is reproduced. An example in which it is synchronized with the NTP time will be described. In this case, both the NTP time on the transmission side and the NTP time on the reception side are adjusted by −1 second when leap seconds are deleted.

99. It is assumed that the NTP time in FIG. 99 is common to the NTP time on the transmission side and the NTP time on the reception side. In the following description, it is assumed that there is no transmission delay. In the following description, only the AU PTS correction method in the MPU will be described. However, the DTS of all AUs in the MPU can be corrected by using the same method. In FIG. 99, it is assumed that the number of AUs included in one MPU is 5, but the number of AUs included in an MPU is not limited to this.

As in the transmission method described with reference to FIG. 85, the NTP time (up to 8:59:58) immediately before the leap second deletion is C area, and the NTP time after the leap second deletion (after 9:00:00) is D. This is an area.

The transmission side (transmission device) performs the following processing.

-Stores the MPU time stamp in the MPU time stamp descriptor and transmits it to the receiving device without correcting the generated MPU time stamp. That is, the first time information is time information that is not corrected on the transmitting side of the MPU during leap second adjustment. Also, identification information indicating whether the timing at which the MPU time stamp is given is the C region or the D region is transmitted to the receiving device. Specifically, an MMTP packet storing a plurality of MPUs is transmitted. The MMTP packet has the same configuration as the MMTP packet described in FIG.

In FIG. 99, the timing at which the MPU time stamp is added is indicated by an arrow.

The receiving device performs the following processing.

Next, the receiving apparatus corrects the PTS for each AU. In the AU to be corrected, the MPU time stamp application timing of the MPU to which the AU belongs (the timing indicated by the arrow in FIG. 99) is the C region, and the PTS for each AU before correction is 8:59: 59 is an AU indicating 59 or later. The AUs to be corrected are AUs indicated by underline in FIG. 99, and are AU # 5 in MPU # 2 and AU # 1-AU # 5 in MPU # 3. When the receiving apparatus determines that the AU is a PTS correction target AU, the receiving apparatus corrects the PTS of the AU by +1 second.

That is, in the correction, for each of a plurality of AUs, the MPU time stamp of the MPU storing the AU is generated based on the reference time information before the leap second adjustment, and the calculated PTS of the AU or It is determined whether or not the DTS satisfies a correction condition after a predetermined time (8:59:59 in the case of leap second deletion). In the correction, the PTS or DTS of the AU determined to satisfy the above correction condition is corrected in the determination.

The receiving device can perform synchronous playback of all AUs based on the NTP time at which leap second deletion was performed and the corrected PTS.

FIG. 100 is a diagram showing an operation flow of the receiving apparatus described with reference to FIGS. 98 and 99.

Note that the operation flow on the transmission side is the same as the operation flow shown in FIG.

The receiving apparatus first calculates time stamps (PTS and DTS) for all AUs using the MPU time stamp descriptor and the MPU extended time stamp descriptor (S2901).

Next, on the basis of the identification information signaled by the transmission side, whether for each AU, the MPU time stamp assignment timing of the MPU to which the AU belongs is the A region, the B region, or the C region. , D is determined (S2902). Here, the A region to the D region are the same as those defined above, and thus description thereof is omitted. As in FIG. 89, the case where the leap second is not inserted or deleted is not shown.

If it is determined in step S2902 that the MPU time stamp of the MPU to which the AU belongs is assigned based on the NTP time of the A area, is the AU time stamp calculated in step S2901 after 9: 00: 0: It is determined whether or not (S2903).

If it is determined that the time stamp of AU is after 9:00: 00 (Yes in step S2903), the time stamp value of AU is corrected by −1 second (S2904).

If it is determined in step S2902 that the MPU time stamp of the MPU to which the AU belongs is given based on the NTP time of the C region, whether or not the AU time stamp calculated in step S2901 is 8:59:59 or later. Is determined (S2905).

If it is determined that the AU time stamp is 8:59:59 or later (Yes in step S2905), the AU time stamp value is corrected by +1 second (S2906).

On the other hand, if it is determined in step S2902 that the MPU time stamp of the MPU to which the AU belongs is given based on the NTP time of the B area or the D area, or the AU time stamp value is not 90000 or later in step S2903. If it is determined (No in step S2903), or if it is determined in step S2905 that the AU timestamp value is not 8:59:59 or later (No in step S2905), the AU timestamp is not corrected ( Step S2907), the operation of the receiving device is terminated.

As described above, as represented by the MPU time stamp descriptor and the MPU extended time stamp descriptor, the first time information indicating the absolute value of the MPU time stamp and the second time indicating the relative value with respect to the absolute value. When the information is transmitted, the transmitting side transmits identification information indicating an NTP time region as a base to which the absolute value of the time stamp is added together with the absolute value of the time stamp.

In the receiving apparatus, after calculating the time stamps (PTS and DTS) of all AUs based on the first time information indicating the absolute value of the time stamp of the MPU and the second time information indicating the relative value, each time stamp, and The time stamp is corrected by determining whether or not to correct the time stamp based on the identification information indicating the area of the NTP time that is the basis of the absolute value of the MPU time stamp that is the base of the time stamp. .

Information indicating whether or not the time stamp is corrected for each AU in the receiving apparatus, and whether the timing at which the MPU time stamp of the MPU to which each AU belongs is the A area, the B area, or the C area The additional information such as the information indicating whether it is the D area does not need to be indicated by all the MPUs, and may be indicated in a section including the MPU including at least the AU to be corrected.

For example, when the time difference between the time stamp (PTS or DTS) in an arbitrary AU and the NTP time to which the MPU time stamp of the MPU to which the AU belongs is N seconds, when leap seconds are inserted or leap When seconds are deleted, the period in which the AU to be corrected exists is at least N seconds before the leap second adjustment. Therefore, the area A at the time of leap second insertion or the area C at the time of leap second deletion needs to be an area indicating a period of at least N seconds before the leap second adjustment.

Conversely, the time indicating the A area at the time of the leap second insertion and the C area at the time of the leap second deletion may be defined as M seconds, and a restriction may be given to always give a time stamp so that N <M.

Further, it is not always necessary to signal the B area at the time of the leap second insertion or the D area at the time of the leap second deletion, and information indicating the transition from the A area to the B area at the time of the leap second insertion. At the time of deletion, it is only necessary to show at least information indicating that the area has shifted from the C area to the D area.

In short, the receiving apparatus first calculates the time stamps of all the access units based on the first time information (absolute time information) and the second time information (relative time information), and sets all the access units. , The time stamp of the access unit is corrected based on the identification information indicating whether the NTP time that is the basis of the calculation of the MPU time stamp of the MPU to which the access unit belongs is before the leap second adjustment. Specifically, when the identification information indicates the time until the leap second is inserted, and when the time stamp of the access unit indicates 9:00 or after, the time stamp is corrected by −1 second. Further, when the identification information indicates the time until the leap second is deleted, and when the time stamp of the access unit indicates 8:59:59 or later, the time stamp is corrected by +1 second. In the case of an access unit whose time stamp is corrected, the receiving apparatus presents the access unit based on the reference time information after the leap second adjustment is performed.

In this way, when the leap second adjustment is performed, it is possible to correct the time stamps of all the access units by determining the timing of giving the MPU time stamps of the MPUs to which the access units belong. Appropriate reception processing using the stamp descriptor becomes possible. That is, even when the leap second adjustment is performed on the reference time information that is the reference of the reference clock of the transmission side and the reception device, a plurality of access units stored in the MPU can be reproduced at the intended time. .

[Supplement: Transmission / reception system and receiver]
As described above, a transmission / reception system including a transmission apparatus that stores and transmits data constituting an encoded stream in an MPU and a reception apparatus that receives the transmitted MPU can be configured as shown in FIG. It is. Further, the receiving apparatus can be configured as shown in FIG. FIG. 101 is a diagram illustrating an example of a specific configuration of the transmission / reception system. FIG. 102 is a diagram illustrating an example of a specific configuration of the reception apparatus.

The transmission / reception system 1100 includes a transmission device 1200 and a reception device 1300.

The transmission apparatus 1200 includes a generation unit 1201 and a transmission unit 1202. Each of the generation unit 1201 and the transmission unit 1202 is realized by, for example, a microcomputer, a processor, or a dedicated circuit.

The generation unit 1201 generates first time information (MPU time stamp) indicating the presentation time of the first data unit (MPU) based on the reference time information (NTP time) received from the outside.

The transmission unit 1202 includes a plurality of second data including a first data unit (MPU), first time information (MPU time stamp) generated by the generation unit 1201, and first time information (MPU time stamp). Second time information (relative information) indicating the presentation time (PTS) or decoding time (DTS) of each unit (AU) and identification information are transmitted.

The receiving device 1300 includes a receiving unit 1301, a calculating unit 1302, and a correcting unit 1303. Each of the reception unit 1301, the calculation unit 1302, and the calculation unit 1303 is realized by, for example, a microcomputer, a processor, or a dedicated circuit.

Detailed description of each component of the receiving device 1300 will be given in the description of the receiving method.

The reception method will be described with reference to FIG. FIG. 103 is an operation flow (reception method) by the reception apparatus.

First, the receiving unit 1301 of the receiving device 1300 receives a first data unit (MPU), first time information (MPU time stamp), second time information (relative information), and identification information (S3001). ).

Next, the calculation unit 1302 of the reception device 1300 uses the first time information (MPU time stamp) and the second time information (relative information) received by the reception unit 1301 to receive a plurality of pieces received by the reception unit 1302. The presentation time (PTS) or decoding time (DTS) of each of the second data units (AU) is generated (S3002).

Next, the correction unit 1303 of the reception device 1300 includes the presentation times (PTS) of each of the plurality of second data units (AU) calculated by the calculation unit 1302 based on the identification information received by the reception unit 1301. The decoding time (DTS) is corrected (S3003).

Thereby, the receiving apparatus 1300 can normally reproduce the plurality of second data units stored in the first data unit even when the leap second adjustment is performed.

In the eighth embodiment, reproduction refers to processing including at least one of decoding and presentation.

(Embodiment 9)
In the eighth embodiment, the method for correcting the PTS or DTS for each AU included in the MPU has been described in the receiving apparatus. In the ninth embodiment, the method for correcting the PTS or DTS in the transmitting apparatus will be described. . That is, a correction method for correcting a time stamp indicating the PTS or DTS of each of a plurality of other AUs excluding the head AU among the plurality of AUs included in the MPU will be described.

FIG. 104 is a diagram for explaining a correction method for correcting a time stamp on the transmission side (transmission device) when a leap second is inserted. FIG. 104 shows the same case as FIG.

104. It is assumed that the NTP time in FIG. 104 is common to the NTP time on the transmission side and the NTP time on the reception side. In the following description, it is assumed that there is no transmission delay. In the following, only the correction method of the AU PTS in the MPU will be described, but the AU DTS in the MPU can also be corrected by using the same method. 104 explains that the number of AUs included in one MPU is 5, but the number of AUs included in an MPU is not limited to this.

The transmission side (transmission device) performs the following processing.

1. As described with reference to FIG. 83 in the sixth embodiment, a generation process for generating an MPU time stamp which is presentation time information (first time information) of an MPU is performed.

2. When the MPU time stamp assignment timing (the timing indicated by the arrow in FIG. 104) is included in the A area and the MPU time stamp (the MPU time stamp value before correction) indicates 9:00 or later The MPU time stamp value is corrected by −1 second, and the corrected MPU time stamp is stored in the MPU time stamp descriptor. That is, when the MPU time stamp is generated based on the NTP time included in the A area and the MPU time stamp indicates 9:00: 00 or later, the MPU time stamp is corrected by −1 second. Here, “9:00: 00” is a time corresponding to Japan time as a reference time for the leap second adjustment (that is, a time calculated by adding 9 hours to UTC time). It is.

Also, when the timing for giving the MPU time stamp (the timing indicated by the arrow in FIG. 104) is the B region, the MPU time stamp is not corrected. That is, when the MPU time stamp is generated based on the NTP time included in the area B, the MPU time stamp is not corrected.

In the MPU extended time stamp descriptor, second time information for calculating the PTS or DTS for each AU is stored as relative information from the PTS of the first AU of the MPU stored in the MPU time stamp descriptor. ing. At this time, the relative information stored in the MPU extended time stamp descriptor is not a value considering the insertion of leap seconds.

Therefore, in the generation process 1 described above, the MPU time stamp is corrected when the MPU time stamp as the first time information satisfies the correction condition. For this reason, the first time information is time information to be corrected during the leap second adjustment. The correction condition is that the MPU time stamp is generated based on the NTP time before the leap second adjustment and is after 9:00: 00.

3. A transmission process for transmitting an MMTP packet storing a plurality of MPUs to the receiving apparatus is performed. The MMTP packet includes, as control information, an MPU time stamp descriptor including an MPU time stamp as first time information, an MPU extended time stamp descriptor including relative information as second time information, and identification information.

Here, the identification information uses the MPU time stamp and relative information when the MPU time stamp does not correspond to the MPU time stamp correction condition in the generation process 1 and the transmitting side does not correct the MPU time stamp. Information for the receiving apparatus to identify an MPU including an AU that requires correction by leap second insertion for the PTS or DTS when the receiving apparatus calculates a PTS or DTS for each AU in the MPU. .

That is, the identification information is an AU that is presented in a PTS that requires correction or a DTS that needs to be corrected by performing leap second adjustment to the reference time information. , Information indicating that there is a possibility that other AUs other than the head AU (hereinafter referred to as “correction required AU”) may be included in the MPU. In other words, the identification information can also be said to be information indicating whether or not the receiving apparatus determines whether the correction required AU is included in the MPU.

The identification information indicates, for example, whether the MPU time stamp is information that is generated based on the NTP time before the leap second adjustment in the generation process 1 and is not corrected in the correction process 2 described above. Information. That is, the identification information includes first identification information indicating whether or not the MPU time stamp is generated based on the NTP time before the leap second adjustment, and second identification information indicating that the MPU time stamp is corrected.

Thus, when the identification information indicates that the MPU time stamp is information that is generated based on the NTP time before the leap second adjustment in the generation process and is not corrected in the correction process, the correction required AU is Indicates that it may be included in the MPU. On the other hand, if the identification information indicates that the MPU time stamp is information that has not been generated based on the NTP time before the leap second adjustment in the generation process or has been corrected in the correction process, the correction required AU Is not included in the MPU (that is, the receiving apparatus is not allowed to determine whether the AU requiring correction is included in the MPU).

The processing on the transmission side (transmission device) will be specifically described with reference to FIG.

In the example shown in FIG. 104, MPU # 2 corresponds to the correction condition in the above correction process 2, and the MPU time stamp of MPU # 2 is corrected by −1 second on the transmission side as indicated by the underline, MPU time stamps of other MPUs are not corrected. The MPU time stamp calculated as described above is shown in the MPU time stamp descriptor. On the other hand, in the MPU extended time stamp descriptor (not shown), the PTS (DTS) for each AU in the MPU is conventionally calculated from the MPU time stamp. Shown by relative information.

Further, MPU # 1 in FIG. 104 is an MPU including a correction required AU. Here, FIG. 104 shows the PTS for each AU calculated based on the MPU time stamp descriptor and the MPU extended time stamp descriptor. Although the PTS of AU # 5 in MPU # 1 is calculated as 9: 00: 00.1, it is actually an AU that should be presented at the second 8:59:59 after insertion of leap seconds, − One second correction is required. That is, AU # 5 in MPU # 1 is a correction-required AU.

As described above, in the transmission apparatus, the timing for applying the MPU time stamp (the timing indicated by the arrow in FIG. 104) is the A region and does not satisfy the correction condition (that is, the MPU time stamp before correction). Is at least one of the PTSs (DTS) of all AUs in the MPU calculated using the MPU time stamp before correction and the relative information before 9:00: 00 as a predetermined time The MPU corresponding to the condition (the above value is after 9:00: 00) can be determined to be the MPU including the correction-required AU, and information indicating that the MPU includes the correction-required AU is received from the transmission apparatus. Signaling to In other words, when the MPU time stamp is not corrected, the MPU does not correct the MPU time stamp, and the AU to be presented at the first 8:59:59 unit and the AU to be presented at the second 8:59:59 unit. It can also be said that MPU has both.

The receiving device performs the following processing.

1. A reception process for receiving the MMTP packet transmitted by the transmission side (transmission device) is performed.

2. Based on the MPU time stamp descriptor and the MPU extended time stamp descriptor included in the control information of the MMTP packet, calculation processing for calculating PTSs (DTS) of all AUs is performed.

3. Based on the identification information signaled by the transmission side (transmitting apparatus) and indicating whether or not the MPU may contain the correction required AU, the identification information indicates that there is a possibility of including the correction required AU. A determination process is performed to determine whether or not a correction-required AU is included for an existing MPU. Specifically, it is determined whether or not PTS (DTS) of all AUs included in the MPU is after 9:00: 00, and AU with PTS (DTS) after 9:00: 00 is corrected. By determining that the AU is required, it is determined whether the MPU includes a correction AU. Then, −1 second correction is performed on the PTS (DTS) of the AU requiring correction.

The processing of the receiving device will be specifically described with reference to FIG.

In FIG. 104, since the transmission side signals that MPU # 1 is an MPU that may include a correction-required AU, the PTS calculated based on the time stamp descriptor in MPU # 1 is 9:00: AU # 5 indicating 00 or later is a correction target, and −1 second correction is performed. Thereby, after correction, the PTS of AU # 5 in MPU # 1 is 8: 59: 59.1.

In addition, when the transmission side signals that the MPU has no possibility of including the correction required AU, the above determination processing is not performed in the reception device. Therefore, for a plurality of AUs stored in the MPU, the PTS (DTS) of the plurality of AUs calculated based on the MPU time stamp descriptor and the MPU extended time stamp descriptor is not corrected. By doing so, the PTSs (DTS) of all AUs can be calculated.

Note that the receiving device corrects either the transmitting device or the receiving device based on information indicating whether the MPU time stamp is corrected on the transmitting side or whether the receiving device corrects the PTS (DTS). When the presentation or decoding is performed using the time stamp, the presentation or decoding is performed based on the NTP time after the leap second insertion is always performed (second time 8:59:59 or later). On the other hand, when the presentation or decoding is performed using the time stamp before correction, the presentation or decoding is performed based on the NTP time before the leap second insertion is performed (first 8:59:59 or earlier). Decrypt. In the example of FIG. 104, AUs up to AU # 4 in MPU # 1 are presented based on the NTP time before the leap second insertion is performed (first time 8:59:59 or earlier), and MPU # 1 The AUs after AU # 5 are presented on the basis of the NTP time after the leap second insertion is performed (the second time after 8:59:59 units).

In the present embodiment, it has been described that the transmitting side stores the relative value from the MPU time stamp without considering the leap second insertion as usual in the MPU extended time stamp descriptor. Therefore, the relative value after correcting the time stamp of AU # 5 in MPU # 1 by −1 second may be stored. At this time, the AU is corrected, and it is separately signaled that the AU is to be presented or decoded based on the NTP time after the leap second is inserted (after the second 8:59:59 seconds).

The receiving device calculates the PTS or DTS of all AUs based on the MPU time stamp descriptor and the MPU extended time stamp descriptor. At this time, since the relative value has already been corrected, it is not necessary to correct the time stamp of the AU at the receiving apparatus, information indicating whether the MPU time stamp has been corrected, and the PTS or DTS of the AU in the MPU. All AUs can be presented or decoded based on the information indicating whether or not is corrected.

FIG. 105 is a diagram for explaining a correction method for correcting a time stamp on the transmission side (transmission apparatus) when leap seconds are deleted. FIG. 105 shows the same case as FIG. 84 (b).

Here, the NTP time on the transmission side is synchronized with the NTP server, and the NTP time on the reception side is reproduced based on the time stamp stored in the NTP packet transmitted from the transmission side, The case where the transmission side is synchronized with the NTP time will be described as an example. In this case, both the NTP time on the transmitting side and the NTP time on the receiving side are adjusted by −1 second when a leap second is inserted.

105. It is assumed that the NTP time in FIG. 105 is common to the NTP time on the transmission side and the NTP time on the reception side. In the following description, it is assumed that there is no transmission delay. In the following, only the correction method of the AU PTS in the MPU will be described, but the AU DTS in the MPU can also be corrected by using the same method. 104 explains that the number of AUs included in one MPU is 5, but the number of AUs included in an MPU is not limited to this.

The transmission side (transmission device) performs the following processing.

2. When the MPU time stamp application timing (the timing indicated by the arrow in FIG. 105) is included in the C area and the MPU time stamp (MPU time stamp value before correction) indicates 8:59:59 or later. The MPU time stamp value is corrected by +1 second, and the corrected MPU time stamp is stored in the MPU time stamp descriptor. That is, when the MPU time stamp is generated based on the NTP time included in the C region and the MPU time stamp indicates 8:59:59 or later, the MPU time stamp is corrected by +1 second. Note that “8:59:59” here is a time corresponding to Japan time as a reference time for the leap second adjustment (that is, a time calculated by adding 9 hours to the UTC time). This is the time obtained by subtracting -1 second from.

Also, when the timing for applying the MPU time stamp (the timing indicated by the arrow in FIG. 105) is the D region, the MPU time stamp is not corrected. That is, when the MPU time stamp is generated based on the NTP time included in the D area, the MPU time stamp is not corrected.

In the MPU extended time stamp descriptor, second time information for calculating the PTS or DTS for each AU is stored as relative information from the PTS of the first AU of the MPU stored in the MPU time stamp descriptor. ing. At this time, the relative information stored in the MPU extended time stamp descriptor is not a value in consideration of leap second deletion.

Therefore, in the generation process 1 described above, the MPU time stamp is corrected when the MPU time stamp as the first time information satisfies the correction condition. For this reason, the first time information is time information to be corrected during the leap second adjustment. The correction condition is that the MPU time stamp is generated based on the NTP time before the leap second adjustment, and is 8:59:59 or later.

Here, the identification information uses the MPU time stamp and relative information when the MPU time stamp does not correspond to the MPU time stamp correction condition in the generation process 1 and the transmitting side does not correct the MPU time stamp. When the receiving device calculates the PTS or DTS for each AU in the MPU, it is information for the receiving device to identify the MPU including the AU that requires correction by leap second deletion for the PTS or the DTS. .

Here, since the identification information is the same as the identification information described in the leap second insertion, the description is omitted.

Processing on the transmission side (transmission device) will be specifically described with reference to FIG.

In the example shown in FIG. 105, the MPU # 3 corresponds to the condition for correcting the MPU time stamp in the above correction process 2, and the MPU time stamp of MPU # 3 is set to +1 on the transmission side as indicated by the underline. Seconds are corrected, and MPU time stamps of other MPUs are not corrected. The MPU time stamp calculated as described above is shown in the MPU time stamp descriptor. On the other hand, in the MPU extended time stamp descriptor (not shown), the PTS (DTS) for each AU in the MPU is conventionally calculated from the MPU time stamp. Shown in relative information.

Also, MPU # 2 in FIG. 105 is an MPU including a correction-required AU.

Here, 105 indicates a PTS for each AU calculated based on the MPU time stamp descriptor and the MPU extended time stamp descriptor. Although the PTS of AU # 5 in MPU # 2 is calculated as 8: 59: 59.1, it is actually an AU that should be presented at 9: 00: 00: 00 after the leap second is deleted, and is corrected by +1 second is required. That is, AU # 5 in MPU # 2 is a correction-required AU.

As described above, in the transmission apparatus, the timing for applying the MPU time stamp (the timing indicated by the arrow in FIG. 105) is the C region and does not satisfy the correction condition (that is, the MPU time stamp before correction). Is at least one of the PTSs (DTS) of all AUs in the MPU calculated using the MPU time stamp and the relative information before the correction, and is before 8:59:59 as a predetermined time. The MPU corresponding to the condition (the above value is 8:59:59 or later) can be determined to be an MPU including a correction-required AU, and information indicating that the MPU includes a correction-required AU is transmitted from the transmission apparatus to the reception apparatus. Signaling to In other words, in the case where the MPU time stamp is not corrected, the MPU has an AU to be presented at 8:59:58 before the leap second deletion in the MPU, and at 9:00: 00 after the leap second deletion. It can also be said that the MPU has both AUs to be presented.

The receiving device performs the following processing.

3. Based on the identification information signaled by the transmission side (transmitting apparatus) and indicating whether or not the MPU may contain the correction required AU, the identification information indicates that there is a possibility of including the correction required AU. A determination process is performed to determine whether or not a correction-required AU is included for an existing MPU. Specifically, it is determined whether or not PTS (DTS) of all AUs included in the MPU is 8:59:59 or later, and AUs whose PTS (DTS) is 8:59:59 or later are corrected. By determining that the AU is required, it is determined whether the MPU includes a correction AU. Then, +1 second correction is performed on the PTS (DTS) of the AU requiring correction.

In FIG. 105, since the transmitting side signals that MPU # 2 is an MPU that may include a correction-required AU, the PTS calculated based on the time stamp descriptor in MPU # 2 is 8:59: AU # 5 indicating 59 or later is a correction target, and +1 second correction is performed. Thereby, after correction, the PTS of AU # 5 in MPU # 2 becomes 9: 00: 00.1.

Note that the receiving apparatus can present or decode the AU using the corrected PTS (DTS).

In the present embodiment, it has been described that, on the transmitting side, the MPU extended time stamp descriptor stores a relative value from the MPU time stamp without considering leap second deletion as before, but correction by leap second deletion is performed. Therefore, the relative value after correcting the time stamp of AU # 5 in MPU # 1 by +1 second may be stored.

The receiving device calculates the PTS or DTS of all AUs based on the MPU time stamp descriptor and the MPU extended time stamp descriptor. At this time, since the relative value is already corrected, it is not necessary to correct the time stamp of the AU in the receiving apparatus, and all AUs can be presented or decoded.

Of the correction methods for leap second insertion or leap second deletion described so far, the transmission side (transmission device) may be able to select which correction method to use. When the transmission side (transmission device) selects a correction method, the transmission side signals to the reception device which method the transmission side has corrected. Based on the signaled information, the receiving device can switch correction, presentation operation, or decoding operation in the receiving device.

FIG. 106 is a diagram showing an operation flow of the transmission apparatus described in FIG. FIG. 107 is a diagram showing an operation flow of the receiving apparatus described in FIG.

In FIG. 106, the transmission apparatus determines whether the timing for giving the MPU time stamp is the A area or the B area (S3101). Here, since the A region and the B region are the same as those defined above, the description thereof is omitted. As in FIG. 89, the case where the leap second is not inserted is not shown.

If it is determined in step S3101 that the area is the B area, the MPU time stamp is not corrected (S3102), and the process ends.

On the other hand, if it is determined in step S3101 that the area is A, it is determined whether or not the MPU time stamp is after 9:00: 00 (S3103).

When it is determined that the MPU time stamp is after 9:00: 00 (Yes in step S3103), the MPU time stamp is corrected by −1 second, and the MPU time stamp is corrected and signaled to the receiving apparatus (S3104). The process is terminated.

On the other hand, when it is determined that the MPU time stamp is not after 9:00: 00 (No in step S3103), all the AUs included in the MPU using the MPU time stamp descriptor and the MPU extended time stamp descriptor are displayed. Time stamps (PTS and DTS) are calculated (S3105).

Next, it is determined whether or not an AU whose calculated time stamp is 9:00 or later is included in the MPU (S3106).

If it is determined that an AU whose calculated time stamp is 9:00: 00 or later is included (Yes in S3106), the receiving device needs to correct the time stamp of the AU in the MPU. Signaling is performed (S3107).

On the other hand, if it is determined that an AU whose calculated time stamp is 9:00 or later is not included in the MPU (No in S3106), the process ends.

107, the receiving apparatus calculates the time stamps of all AUs using the MPU time stamp descriptor and the MPU extended time stamp descriptor (S3201).

Next, based on the signaled identification information, it is determined whether or not it is necessary to correct the time stamp of the AU in the MPU (S3202).

If it is determined that the time stamp of the AU in the MPU needs to be corrected (Yes in step S3202), the time stamp of the AU indicating that the time stamp is after 9:00: 00 is corrected by −1 second (S3205).

On the other hand, if it is determined that it is not necessary to correct the time stamp of the AU in the MPU (No in step S3202), it is determined whether the MPU time stamp is corrected based on the signaled identification information (S3203). ).

If it is determined that the MPU time stamp has not been corrected (No in step S3203), the time before the time stamp to be corrected is presented with the AU based on the NTP time before the leap second insertion, and the time to be corrected For the time stamp after the stamp, AU is presented based on the NTP time after the leap second insertion (S3204).

After step S3205 or when it is determined in step S3203 that the time stamp has been corrected (Yes in step S3203), the AU is presented based on the NTP time after the leap second insertion is performed (S3206).

FIG. 108 is a diagram showing an operation flow of the transmission apparatus described in FIG. FIG. 109 is a diagram showing an operation flow of the receiving apparatus described in FIG.

108, the transmitting apparatus determines whether the timing for applying the MPU time stamp is the C area or the D area (S3301). Here, since the C region and the D region are the same as those defined above, description thereof is omitted. As in FIG. 89, the case where the leap second is not deleted is not shown.

If it is determined in step S3301 that it is the D region, the MPU time stamp is not corrected (S3302), and the process is terminated.

On the other hand, if it is determined in step S3301 that the area is the C area, it is determined whether or not the MPU time stamp is 8:59:59 or later (S3303).

If it is determined that the MPU time stamp is 8:59:59 or later (Yes in step S3303), the MPU time stamp is corrected by +1 second (S3304), and the process ends.

On the other hand, if it is determined that the MPU time stamp is not after 8:59:59 (No in step S3303), all the AUs included in the MPU using the MPU time stamp descriptor and the MPU extended time stamp descriptor are displayed. Time stamps (PTS and DTS) are calculated (S3305).

Next, it is determined whether or not an AU whose calculated time stamp is 8:59:59 or later is included (S3306).

If the calculated time stamp is 8:59:59 or later but it is determined that AU is included (Yes in S3306), it is determined that the receiving device needs to correct the time stamp of AU in the MPU. (S3307).

On the other hand, if it is determined that an AU whose calculated time stamp is 8:59:59 or later is not included in the MPU (No in S3306), the process is terminated.

109, the receiving apparatus calculates the time stamps of all AUs using the MPU time stamp descriptor and the MPU extended time stamp descriptor (S3401).

Next, based on the signaled identification information, it is determined whether or not it is necessary to correct the time stamp of the AU in the MPU (S3402).

If it is determined that the AU time stamp in the MPU needs to be corrected (Yes in step S3402), the AU time stamp indicating the time stamp after 8:59:59 is corrected by +1 second (S3403).

After step S3403 or when it is determined in step S3402 that it is not necessary to correct the time stamp of the AU in the MPU (No in step S3402), the AU is presented based on the NTP time.

According to this, (i) the MPU, (ii) the MPU time stamp, and (iii) the second data unit that is presented at the presentation time that needs to be corrected or decoded at the decoding time that needs to be corrected An identification indicating that a second data unit other than the first second data unit in the presentation order among the plurality of second data units may be included in the first data unit Send information and. For this reason, even when the leap second adjustment is performed on the reference time information that is the reference of the reference clocks of the transmission side and the reception device, the reception device has a plurality of second data stored in the first data unit. Two data units can be reproduced at the intended time.

[Supplement: Transmitter and receiver]
As described above, the transmission apparatus that stores the data constituting the encoded stream in the MPU and transmits the data can also be configured as shown in FIG. Also, a receiving apparatus that receives an MPU in which data constituting an encoded stream is stored can be configured as shown in FIG. FIG. 110 is a diagram illustrating an example of a specific configuration of the transmission apparatus. FIG. 111 is a diagram illustrating an example of a specific configuration of the reception apparatus.

The transmission apparatus 1400 includes a generation unit 1401 and a transmission unit 1402. Each of the generation unit 1401 and the transmission unit 1402 is realized by, for example, a microcomputer, a processor, or a dedicated circuit.

The receiving device 1500 includes a receiving unit 1501 and a determining unit 1502. Each of the reception unit 1501 and the determination unit 1502 is realized by, for example, a microcomputer, a processor, or a dedicated circuit.

Detailed description of each component of the transmission device 1400 and the reception device 1500 will be given in the description of the transmission method and the reception method, respectively.

First, the transmission method will be described with reference to FIG. FIG. 112 is an operation flow (transmission method) by the transmission apparatus.

First, the generation unit 1401 of the transmission device 1400 generates first time information indicating the presentation time of the MPU as the first data unit] based on the reference time information received from the outside (S3501).

Next, the transmission unit 1402 of the transmission device 1400 transmits a first data unit (MPU), first time information (MPU time stamp), and identification information (S3502). Here, the first time information (MPU time stamp) is information generated by the generation unit 1401. The identification information is a second data unit (access unit) that is presented at a presentation time (PTS) that needs to be corrected or decoded at a decoding time (DTS) that needs to be corrected. Other second data units (access units) other than the first second data unit (access unit) in the presentation order of the second data units (access units) are included in the first data unit (MPU). This information indicates that there is a possibility that

Thus, the receiving device that has received the information transmitted from the transmitting device 1400 can reproduce the AU in the MPU based on the identification information even when the leap second adjustment is performed. Can be played back at the intended time.

Next, the reception method will be described with reference to FIG. FIG. 113 is an operation flow (reception method) by the reception apparatus.

First, the receiving unit 1501 of the receiving device 1500 receives the first data unit (MPU), the first time information (MPU time stamp), and the identification information (S3601). The first time information and the identification information are the same as the first time information and the identification information described in the transmission apparatus 1400.

Next, the determination unit 1502 of the reception device 1500 includes the first data indicated by the identification information received by the reception unit 1501 to indicate that another second data unit (access unit) may be included. It is determined whether another second data unit (access unit) is included in the unit (MPU) (S3602).

Thereby, the receiving apparatus 1500 can reproduce the AU in the MPU at the intended time even when the leap second adjustment is performed.

(Embodiment 10)
FIG. 114 is a diagram showing details of the protocol stack diagram of the MMT / TLV system defined in ARIB STD-B60, and is a diagram detailing FIG.

In the MMT / TLV system, video, audio, caption / superimpose, application, etc. are transmitted as MMTP packets. The caption / superimpose is TTML encoded. As an application, an HTML5 application is stored in an MMTP packet.

FIG. 115 is a block diagram of the receiving apparatus, and is a more detailed view of FIG.

The receiving apparatus 1600 extracts the TLV packet by decoding the transmission line encoded data in the demodulating means 1602 with respect to the broadcast signal received by the tuner 1601 and performing error correction or the like on the data obtained by the decoding. .

TLV / IP / UDP DEMUX means 2603 performs TLV DEMUX processing and IP / UDP DEMUX processing. The TLV / IP / UDP DEMUX means 2603 performs processing according to the data type in the TLV DEMUX processing. For example, in the case of a compressed IP packet, the compressed header is restored. The TLV / IP / UDP DEMUX means 2603 processes TLV-SI (AMT, NIT) when the data type is TLV-SI. The TLV / IP / UDP DEMUX unit 2603 performs processing such as discarding when the data type is a null packet. In the IP / UDP DEMUX process, the TLV / IP / UDP DEMUX means 2603 performs processing such as header analysis of IP packets and UDP packets, and extracts MMTP packets and NTP packets.

The descrambling means 1604 releases the scramble when the MMTP packet is scrambled. The descrambling means 1604 obtains scramble key information for descrambling from the EMM section, ECM section, etc. in the control information analysis means 1605. When descrambling means 1604 is scrambled in units of IP packets, it descrambles in units of IP packets.

The NTP clock generation unit 1606 regenerates the NTP clock from the NTP packet.

MMT DEMUX means 1607 performs filtering processing of components and control information such as video, audio, subtitles, text super, and application based on the packet ID stored in the MMTP packet header. The MMT DEMUX means 1607 sends a time stamp descriptor stored in the MP table from the control information analysis means 1605 (when describing both the MPU time stamp descriptor and the MPU extended time stamp descriptor, the time stamp description The PTS / DTS calculating means calculates PTS and DTS for each access unit.

The media processing unit 1608 converts each of the filtered video, audio, subtitle, text super, application, etc. from the MMTP packet into unit data. The units to be presented are specifically NAL units and access units of video signals, audio frames, subtitle presentation units, and the like.

The decryption presentation unit 1609 decrypts and presents the access unit at the time when the PTS / DTS of the access unit matches based on the time information of the NTP clock. In addition, when each processing unit is implemented as a different LSI, a different device, or a different device (for example, a television, a display, a recording device, a set-top box, etc.), the decryption and presentation unit 1609 is an external output unit or a communication unit. Each processing block is connected using the interface means 1610 or the like.

Note that the configuration of receiving apparatus 1600 is not limited to the configuration shown in FIG. IP multiplexing systems include DASH / ROUTE system and TS over IP system in addition to MMT. Broadcast transmission systems corresponding to IP multiplexing systems include DVB-T2, DVB-S2, and DVB-C2. , ATSC 3.0. In addition to the TLV method, the Layer 2 protocol serving as an interface between the IP multiplexing method and the broadcast transmission method includes a GSE method, an ALP (ATSC Link-Layer Protocol) method, and the like.

In addition, there are interface standards, communication standards, communication multicasts, etc. using IP multiplexing. Also in these systems, the receiving apparatus has the same configuration as that in FIG.

FIG. 116 is a diagram showing a general broadcast protocol multiplexed by MPEG-2 TS Systems (hereinafter referred to as “TS system”).

In the TS system, video, audio, subtitles / superimposed characters, and applications are stored in PES or section format and transmitted as TS packets.

In conventional ARIB terrestrial digital broadcasting and BS digital broadcasting, caption / text supervision is encoded with ARIB caption. In addition, the application is data broadcast BML encoded.

FIG. 117 is a diagram showing a block diagram of a receiving apparatus that receives broadcast signals multiplexed by the TS method.

The receiving device 1700 extracts TS packets by decoding the transmission path encoded data in the demodulation unit 1702 with respect to the broadcast signal received by the tuner 1701, and performing error correction or the like on the data obtained by the decoding. .

The descrambling means 1703 releases the scramble when the TS packet is scrambled. The descrambling means 1703 acquires scramble key information for descrambling from the EMM section, ECM section, etc. in the control information analysis means 1704.

The reference clock generation means 1705 extracts the PCR from the TS packet and regenerates the PCR clock.

The TS DEMUX means 1706 performs filtering processing of components and control information such as video, audio, subtitles, character super, and applications based on PID (Packet Identifier) stored in the TS packet header.

The media processing means 1707 converts each of the filtered video, audio, subtitle, character super, application, etc. from the TS packet into unit data, and further extracts a time stamp (PTS, DTS) from the PES packet header. The unit to be presented is a video signal NAL unit, access unit, audio frame, caption presentation unit, or the like.

Decoding and presenting means 1708 decodes and presents the access unit at the time when the PTS / DTS of the access unit matches based on the time information of the PCR clock. When each processing unit is mounted as a different LSI, a different device, or a different device (for example, a television, a display, a recording device, a set top box, etc.), an external output means, a communication means, an interface means 1709, etc. To connect each processing block.

Note that the configuration of receiving apparatus 1700 is not limited to the configuration shown in FIG. As transmission systems corresponding to the TS system, there are transmission systems for satellite broadcasting, terrestrial broadcasting, and cable broadcasting standardized by DVB, ISDB, ATSC, etc., interface standards, communication standards, communication multicast systems, and the like. Also in these systems, the receiving apparatus has the same configuration as that in FIG.

There are two types of broadcast or communication multicast systems: a system based on the IP multiplexing system described in FIGS. 114 and 115 and a system based on the TS system described in FIGS. 116 and 117. . The receiving apparatus includes a receiving apparatus corresponding to a single type of receiving process, a receiving apparatus corresponding to a plurality of types of receiving processes, and examples thereof are shown below.

118A, FIG. 118B, FIG. 118C, FIG. 118D, FIG. 118E, and FIG. 118F are configuration examples of the receiving apparatus. The first method is, for example, the MMT / TLV method described in FIG. 114 and FIG. For example, the TS scheme (second multiplexing scheme) described with reference to FIGS. 116 and 117 is used. The first method and the second method may be reversed, or three or more methods may be mixed. The processing means A and the processing means B are processing means in which some functions of the plurality of processing means described with reference to FIGS. 115 and 117 are integrated. In this example, the processing means A and the processing means B are broadly divided. However, any of the processing means described with reference to FIGS. 115 and 117 may be combined, or FIG. In addition, among the plurality of processing means described with reference to FIG. 117, a processing means obtained by further dividing the processing of one processing means may be combined, or three or more processing means may be formed by combining a plurality of processing means. The processing means may be configured.

FIG. 118A is a diagram showing a configuration of a conventional receiving apparatus when processing a single method. In this case, the receiving apparatus processes the signal transmitted by the first method by the processing unit A 1811 of the first method, and then processes the signal by the processing unit B 1812 of the first method.

FIG. 118B is a diagram illustrating a configuration of a conventional receiving apparatus that independently processes a plurality of schemes. In this case, the receiving apparatus processes the signal transmitted by the first method by the processing unit A1821 of the first method, then processes the signal by the processing unit B1822 of the first method, and transmits the signal by the second method. The signal is processed by the second processing means A 1823 and then processed by the second processing means B 1824.

On the other hand, the configuration in the case of using the method conversion means as the conversion unit is shown below.

FIG. 118C is a diagram illustrating an example of a configuration of a receiving device when using a method conversion unit. In this case, the receiving apparatus processes the signal transmitted by the first method by the processing unit A1831 of the first method, and then converts the output of the first method processing unit A1831 to the second method by the method conversion unit 1832. Conversion is performed, and the subsequent processing is processed by the second method processing means B1833.

FIG. 118D is a diagram illustrating another example of a configuration of a receiving device when using a method conversion unit. In this case, the receiving apparatus processes the signal transmitted by the first method by the processing unit A1841 of the first method, and then processes a part of the output of the processing unit A1841 of the first method by the first method. The processing is performed by the means B 1842, the remaining part is converted into the second system by the system conversion means 1843, and the subsequent processing is processed by the second system processing means B 1844.

FIG. 118E is a diagram illustrating another example of the configuration of the reception device when using the method conversion unit. In this case, the receiving apparatus receives the signal transmitted by the first method and the signal transmitted by the second method, and processes the signal transmitted by the first method by the processing means A1851 of the first method. Then, the output of the first method processing means A1851 is converted into the second method by the method conversion means 1852, and the subsequent processing is processed by the second method processing means B1854. On the other hand, the receiving apparatus processes the signal transmitted by the second method by the processing unit A1853 of the second method, and then processes the output of the second method processing unit A1853 by the second method processing unit B1854. . That is, the receiving apparatus processes both the signal transmitted by the first scheme and the signal transmitted by the second scheme by the processing means B of the second scheme.

FIG. 118F is a diagram illustrating another example of the configuration of the reception device when using the method conversion unit. In this case, the receiving apparatus receives the signal transmitted by the first scheme and the signal transmitted by the second scheme, and processes the signal transmitted by the first scheme by the processing unit A1861 of the first scheme. After that, a part of the output of the processing unit A1861 of the first method is processed by the processing unit B1862 of the first method, and the remaining part is converted into the second method by the method conversion unit 1863, and the subsequent processing Is processed by the second system processing means B1865. On the other hand, the receiving apparatus processes the signal transmitted by the second method by the processing unit A1864 of the second method, and then processes the output of the second method processing unit A1864 by the second method processing unit B1865. . That is, the receiving apparatus processes a part of the signal transmitted by the first scheme and the signal transmitted by the second scheme by the processing unit B1865 of the second scheme.

As shown in FIG. 118D or FIG. 118F, when a signal transmitted by the first method is processed by both the first method processing means B 1842 and 1862 and the second method processing means

B

1844 and 1865, Information for synchronization (for example, time information and clock information) may be shared between the first method processing means B1842 and 1862 and the second method processing means B1844 and 1865. That is, in this case, an adjustment may be made between the first method processing means B1842 and 1862 and the second method processing means B1844 and 1865 to match one process with the other process.

For example, the first method processing means A 1831, 1851, 1861 and the second method processing means A 1853, 1864 that perform the first stage processing in FIGS. 118C to 118 F receive the broadcast signal in which the multiplexed data is modulated, and receive it. The first processing unit performs processing for demodulating the broadcast signal and outputting multiplexed data obtained by demodulating. The multiplexed data referred to here is at least the first multiplexed data of the first multiplexed data of the first multiplexing system and the second multiplexed data of the second multiplexing system different from the first multiplexing system. It is data that is composed.

The

system conversion units

1832, 1843, 1852, and 1863 change the multiplexing system of the first multiplexed data out of the multiplexed data output from the first system processing

unit A

1831, 1851, and 1861 as the first processing unit. It is a conversion unit that converts to a multiplexing system and outputs conversion data obtained by conversion.

Also, for example, the second method processing means

B

1833, 1844, 1854, 1865, which perform processing after the method conversion means 1832, 1843, 1852, 1863 in FIGS. 118C to 118F, are the method conversion means 1832, 1843, 1852, The second processing unit performs a decoding process for decoding the conversion data output by 1863 and outputs the decoded data obtained by the decoding process.

The receiving apparatus having the above-described method conversion means 1832, 1843, 1852, 1863 has the following effects.

For example, by including the

method conversion units

1832, 1843, 1852, and 1863, it is not necessary to perform all of the subsequent processing by the first method processing unit B, and at least part of the subsequent processing is performed by the second method processing unit. Can be realized. For example, when only the second method processing means B is provided as an existing implementation, the first method processing means B may not be newly installed. Alternatively, for example, when the second system processing means B is a device (hardware or software) having high performance, the second system processing means B converts only the signal that requires high-performance processing by using the system conversion means. By processing only signals that do not require high-performance processing by the first method processing unit B, the first method processing unit B does not require high performance. For this reason, the circuit scale can be reduced and the cost can be reduced. Here, the second method processing means B that requires high-precision processing is implemented using hardware, and the first method processing means B that does not require high-performance processing is the second method processing means B. It is possible to implement using hardware having only the function of the second method processing means B. In addition, since various configurations can be flexibly realized, it is possible to mount effectively for compatibility with existing systems, circuit scale, and cost reduction.

FIG. 119 is a diagram showing a modified example of the configuration of the receiving apparatus when using the method conversion means.

Here, description will be made assuming that the first method is the MMT / TLV method and the second method is the TS method.

As shown in FIG. 119, the receiving apparatus 1900 includes first method processing means A 1910, method conversion means 1920, and method processing means B 1930.

The first method processing means A 1910 receives an MMT / TLV method signal, performs processing such as TLV Demux, etc., and sends a part of the signal of the MMTP packet to the first method conversion means 1921 as the first conversion unit. The remaining part of the signal is output to the second system conversion means 1922 as the second conversion unit.

The system conversion unit 1920 includes a first system conversion unit 1921, a second system conversion unit 1922, and a second system multiplexing unit 1923 as a multiplexing unit. The first scheme conversion means 1921 stores the MMTP packet data structure in the TS packet and outputs it to the second scheme multiplexing means 1923. In other words, the first scheme conversion means extracts a part of the first multiplexed data and uses the first packet (MMTP packet) constituting the first data in the second multiplexing scheme. The first conversion stored in the second packet (TS packet) is performed, and first conversion data composed of the second packet obtained by the first conversion is output.

Also, the second system conversion means 1922 extracts data from the other MMT packet, converts it into a TS packet, and outputs it to the second system multiplexing means. That is, the second system conversion unit 1922 extracts the first packet (MMTP packet) that constitutes the remaining second data of the first multiplexed data, and performs the second multiplexing on the extracted first packet. The second conversion to be converted into the second packet (TS packet) of the method is performed, and the second conversion data constituted by the second packet obtained by the second conversion is output.

Here, the first data and the second data may be data of different media. For example, the first data may be data indicating video / audio, and the second data may be data indicating subtitles, applications, and the like.

Then, the second system multiplexing unit 1923 multiplexes and outputs the TS packet format data output by the first system conversion unit 1921 and the second system conversion unit 1922 using the TS system. That is, the second system multiplexing means 1923 performs a multiplexing process for multiplexing the output first converted data and second converted data.

As described above, the system conversion unit 1920 outputs a signal through one interface by multiplexing using one of the first system and the second system. That is, the system conversion unit 1920 outputs the data obtained by the multiplexing process as converted data.

The second method processing means B 1920 includes second method demultiplexing means 1931 as a demultiplexing unit, first method processing means B 1932, and second method processing means B 1933. The second system demultiplexing means 1931 receives a signal multiplexed by the TS system and demultiplexes it. That is, the second system demultiplexing means 1931 performs a demultiplexing process for demultiplexing the converted data output by the system converting means 1920 into the first converted data and the second converted data.

Also, the second system demultiplexing processing means 1931 constitutes the first converted data obtained by demultiplexing, and converts the TS packet converted by the first system converting means 1921 into an MMTP packet. The data is converted and output to the first method processing means B. Further, the second system demultiplexing processing means 1931 constitutes the second converted data obtained by demultiplexing, and the TS packet converted by the second system converting means 1922 is converted into the second packet. Output to the system processing means B 1933.

Then, the first method processing means B 1932 processes the MMTP packet. Specifically, the first method processing means B 1932 is configured by the first packet (MMTP packet) extracted from the second packet (TS packet) constituting the first converted data obtained by the demultiplexing process. A first decoding process using the first multiplexing method is performed on the first data.

Also, the second method processing means B 1933 processes the TS packet. Specifically, the second system processing means B 1933 performs the second multiplexing on the second data constituting the second packet (TS packet) constituting the second converted data obtained by the demultiplexing process. The second decoding process is performed using the conversion method.

The method processing unit B1930 uses the first control information of the first decode data and the second control information of the second decode data to adjust the one to the other of the first control information and the second control information. By performing this, an adjustment unit that adjusts the first decode data and the second decode data may be provided. Specifically, the first control information is first reference clock information, the second control information is second reference clock information, and the adjustment unit is one of the first reference clock information and the second reference clock information. The first decode data and the second decode data are synchronized by adjusting the other to the other.

As described above, the system processing means B 1930 may output the first decoded data obtained by the first decoding process and the second decoded data obtained by the second decoding process as decoded data.

FIG. 120 is a diagram showing an example of details of the method conversion means.

As shown in FIG. 120, the first scheme processing means A2010 includes a tuner 2011, a demodulation means 2012, a TLV / IP / UDP Demux means 2013, a first scheme descramble means 2014, a control information analysis means 2015, a first information Reference clock generation means 2016, MMT demux means 2017, and the like. The tuner 2011, demodulation means 2012, TLV / IP / UDP Demux means 2013, first method descramble means 2014, control information analysis means 2015, first reference clock generation means 2016, MMT Demux means 2017 are shown in FIG. This corresponds to the tuner 1601, demodulation unit 1602, TLV / IP / UDP Demux unit 1603, descrambling unit 1604, control information analysis unit 1605, NTP clock generation unit 1606, and MMT Demux unit 1607 of the receiving apparatus 1600 described in the above. Description is omitted.

The system conversion means 2020 includes media conversion means, media conversion means (first media conversion means 2021 and second media conversion means 2022), clock generation means (second reference clock generation means 2023 and difference information generation means 2024), The second scheme multiplexing means 2025, the second scheme scrambling means 2026, etc. are included.

The system conversion means 2020 may use different media conversion means for each medium. For example, the method conversion unit 2020 uses the second medium conversion unit 1922 of the method conversion unit 1920 described in FIG. 119 as the first medium (for example, video and audio), and the second medium (for example, subtitles). May be configured to use the first system conversion unit 1921 of the system conversion unit 1920.

Further, the system conversion means 2020 generates and multiplexes the difference information between the first reference clock information and the second reference clock in the reference clock information conversion means and the difference information generation means. In other words, the method conversion unit 2020 may include the adjustment unit described above.

Hereinafter, a specific example of a method conversion means for converting a signal received by the MMT / TLV method into the TS method will be described.

(Video, audio)
A case where the method conversion means converts the video data and audio data from the first method to the second method will be described.

In the case of conversion by the second system conversion means 1922, the video signal and audio signal stored in the input MMTP packet are converted into access unit units, an elementary stream (ES) is generated, and then a PES header is added. To repacketize. Specifically, the second method conversion unit 1922 extracts the data stored in the MMTP packet, divides the data when the MFU is aggregated, and receives all the data when the MFU is fragmented. Join.

When the MFU of the video signal is in units of NAL units, a plurality of NAL units are further combined to form units of access units. Here, when the video signal is transmitted with the asset type 'hvc1', a non-VCL NAL unit is acquired from the MFU to form an access unit, and when the video signal is transmitted with the asset type 'hev1' Then, a non-VCL NAL unit is acquired from MPU metadata transmitted separately to configure an access unit. Further, in the case of the NAL size format, the 4-byte NAL size area is deleted, and a byte start code is added to convert it into a byte stream and output it.

If the MFU of the audio signal is in units of frames, the MFU is in units of access units. When the MFU is in the LATM / LOAS stream format and in the AudioMuxConfig () format, it may be converted to the AudioSyncStream () format by adding synchronization information and length information, or in the case of the Raw data stream format. May separately acquire AudioMuxConfig () from MPU metadata or control information and convert it to the LATM / LOAS stream format. You may convert into other stream formats.

(Time stamp information)
A case where the method conversion unit converts the time stamp information from the first method to the second method will be described.

Time stamp information (decoding time or presentation time, or information for calculating the decoding time and presentation time) such as video and audio is acquired from the received MMTP packet and converted to PTS and DTS for each access unit. For example, in the control information analysis means, the presentation time of the head access unit is obtained from the MPU time stamp descriptor in the order of presentation in the MPU, and further, the PTS of all access units in the MPU is used together with the MPU extended time stamp descriptor. And DTS. The calculated DTS and PTS are stored in the header of the PES packet including the corresponding video or audio access unit, and the PES packet is generated and output. The first method conversion means 1921 may store and output the MMTP packet storing the synchronous MPU / MFU in the PES packet or TS packet without converting into the PES packet format. As a transmission method at the time of output, it may be stored in a PES packet as a private stream, may be stored and transmitted in a private area of a TS packet, may be sectioned as control information, or may be converted into an IP packet. The data may be stored and transmitted by Ether, or any transmission means such as transmission using a communication means such as USB or I2C.

(Reference clock information)
A case where the system conversion means converts the reference clock information from the first system to the second system will be described. For example, operations of the first reference clock generation unit 2016, the second reference clock generation unit 2023, and the difference information generation unit 2024 illustrated in FIG. 120 will be described.

In the MMT / TLV method, the time stamp is based on a transmission system clock synchronized with UTC, and a time stamp value based on the system clock is separately stored in an NTP packet as reference clock information and transmitted to notify the system clock information. The The receiving apparatus regenerates the receiver system clock based on the reference clock information stored in the NTP packet in the first reference clock generation means 2016 of the first scheme processing means A2010. The system conversion means 2020 converts the system clock information (first reference clock information) reproduced based on the NTP packet into STC (second reference clock information), and uses the time stamp (PCR) as an adaptation field of the TS packet. The time stamp value of the NTP packet may be directly converted into a PCR value, stored in the TS packet, and output. For example, the system conversion means 2020 converts NTP having a power of 2 precision to PCR having a precision of 27 MHz. Further, for example, the method conversion unit 2020 extracts a specific 33 bits from a 64-bit NTP and sets it as a PCR.

(Leap second adjustment of reference clock information)
Next, a leap second adjustment method in the case where the reference clock information stored in the NTP packet is adjusted for leap seconds will be described. When the leap_indicator indicating leap second adjustment is indicated in the NTP packet, the receiving apparatus corrects the PTS and DTS based on mpu_presentation_time_leap_indicator stored in the MPU extended time stamp descriptor, and then corrects the corrected PTS and DTS to the PES packet header. To store. Alternatively, mpu_presentation_time_leap_indicator information may be separately generated and output without correcting the PTS and DTS, or information indicating that leap second correction is necessary for the PTS and DTS for each access unit may be generated and output. Good. The system processing unit B 1930 that has received the PTS and DTS that are not subjected to leap second correction corrects the PTS and DTS based on information indicating that leap second correction is necessary. At the time of the leap second adjustment, the receiver system clock may also perform the leap second adjustment of ± 1 second at the same time, or the time stamp may be corrected without performing the leap second adjustment. If a ± 1 second discontinuity occurs in the STC during leap second adjustment, the PCR discontinuity flag in the TS packet header is validated. The receiving device may independently generate a 27 MHz system clock (STC) and transmit the time stamp value as PCR. In that case, video and audio PTSs and DTSs are assigned based on the uniquely generated STC and stored in the PES packet header. Note that even if a leap_indicator indicating leap second adjustment is indicated in the NTP packet, the PTS and DTS based on the system clock generated independently need not perform leap second adjustment.

If each of the first method processing means B 1932 and the second method processing means B 1933 is provided as in the method processing means B 1930 in FIG. 119, the receiving device generates the NTP clock generated based on the NTP packet independently. There may be two system clocks (STC), and the difference information generation means may generate difference information indicating the time correspondence between the clocks. In this case, the generated difference information and the time stamp (PCR) based on the uniquely generated STC are multiplexed and output. The system processing means B 1930 receives the PCR and reproduces the STC. Since PTS and DTS based on STC (second reference clock information) are assigned to the media processed by the second system processing means B 1933, decoding is performed at a timing when the PTS and DTS coincide with the second reference clock information. Or the presentation is carried out. On the other hand, since the PTS and DTS based on NTP (first reference clock information) are assigned to the media processed by the first method processing means 1910, it corresponds to the second reference clock information based on the difference information. Conversion to time is performed, and decoding or presentation is performed at a timing that matches the second reference clock information. When the leap second adjustment is performed on the NTP clock, a leap_indicator indicating that the leap second adjustment is performed may be output together with the difference information.

As a transmission method at the time of output, it may be stored in a PES packet as a private stream, may be stored in a private area of a TS packet and transmitted, may be sectioned as control information, It may be stored in a packet and transmitted by Ether, or any transmission means such as transmission using a communication means such as USB or I2C. In these cases, an identifier that can be identified as information indicating a time correspondence relationship between clocks or information indicating a leap second correction flag is provided and output. Note that there is a leap second flag in the reference_start_time in the caption and the UTC-NPT reference descriptor in the event message, and leap second information and difference information are generated and output using the same method.

When generating the PES packet, if the data reception timing of the access unit and the time stamp information of the access unit is different, either one of the data is accumulated, and the PES packet is generated when both data are collected. . For example, when it is guaranteed that the time stamp information of an access unit can be received N seconds before the decoding time of the access unit, the access unit is accumulated until N seconds before the decoding time of the access unit, A PES packet is generated and output N seconds before the decoding time of the access unit.

(subtitles)
A case will be described in which the method conversion unit converts the first method to the second method by storing the caption / superimpose stored in the input MMTP packet in the TS packet.

The input data has a format in which, for example, caption data encoded by TTML is stored in the synchronous MPU / MFU and stored in the MMTP packet. The second system conversion means 1922 extracts TTML data from the MMTP packet and converts it into a section format in the TS system. The section format is output in the data carousel method.

1st system conversion means 1921 stores the MMTP packet in which synchronous MPU / MFU is stored in a PES packet or TS packet, without converting into TS section format, and outputs it. As a transmission method at the time of output, it may be stored in a PES packet as a private stream, may be stored and transmitted in a private area of a TS packet, may be sectioned as control information, or may be converted into an IP packet. The data may be stored and transmitted by Ether, or any transmission means such as transmission using a communication means such as USB or I2C.

(application)
A case will be described in which the method conversion unit converts the application data stored in the input MMTP packet from the first method to the second method by storing the application data in the TS packet.

The input data has a format in which, for example, application data encoded in HTML5 is stored in an asynchronous MPU / MFU and stored in an MMTP packet. The second system conversion means 1922 extracts HTML data from the MMTP packet and converts it into a section format in the TS system. The section format is output in the data carousel method.

1st system conversion means 1921 stores the MMTP packet in which asynchronous MPU / MFU is stored in a PES packet or TS packet, without converting into TS section format, and outputs it. As a transmission method at the time of output, it may be stored in a PES packet as a private stream, may be stored and transmitted in a private area of a TS packet, may be sectioned as control information, or may be converted into an IP packet. The data may be stored and transmitted by Ether, or any transmission means such as transmission using a communication means such as USB or I2C.

Note that control information related to application transmission (for example, MH-AIT, application control information descriptor, data transmission message, etc.) is also converted into TS format and output. The control information input in the MMT message format may be converted into the TS section format and output, or the control information input in the MMT message format may be stored as a private stream in the PES packet. The data may be stored in a private area and transmitted. Only the M2 section message may be converted to the TS section format and output, and other messages may be output without being converted to the TS section format.

(Control information)
Control information serving as an entry point for services includes PAT and PMT in the TS system, and PLT, MPT, and AMT in the MMT / TLV system.

When there is a component that uses the second system conversion means 1922, the control information is converted so as to include the information of the component. For example, when video and audio are converted using the second system conversion unit 1922 and captions and applications are converted using the first system conversion unit 1921, service information, video, and audio are converted from the MPT. Are converted into PMT. At the same time, a PAT indicating the PMT location information is generated and output.

If there is a component that uses the first method conversion means 1921, the control information is stored in the PES packet or TS packet and output.

(Packet header conversion)
The header of the MMTP packet includes a packet counter indicating the continuity of the MMTP packet, a packet sequence number indicating the continuity for each asset, and a fragment counter and a fragment indicator indicating the continuity of the divided MFU. When these continuity fields shown in the input MMTP packet are monitored and a packet loss due to a transmission error or the like is detected, the continuity index in the TS packet is set to a discontinuous value. Output. Note that the transport error indicator in the TS packet may be set to 1 to generate and output invalid data, or a null TS packet may be inserted and output to match the transmission rate. Alternatively, when packet loss occurs, data may be restored by, for example, copying previous data. The restored data is stored in the PES packet and TS packet and output. In this case, a continuous value is set and output as the continuity index.

The packet ID (16 bits) indicated in the MMTP packet is converted to the PID (13 bits) of the TS packet. For example, a rule or table that associates the packet ID and PID usable area is prepared in advance, and conversion is performed according to the table. In the case where the MMTP packet is stored in the PES packet or TS packet as private data for each MMTP packet header using the first method conversion means 1921, the TS packet can be used regardless of what data is stored in the MMTP packet. The PID is assigned with a PID indicating that the data is stored as it is while maintaining the data structure of the MMTP packet. When receiving the output TS packet, if the TS packet is given a PID indicating that the data structure of the MMTP packet is stored, the MMTP packet is extracted from the TS packet, and the MMTP Demux process To implement.

In the case of video and audio, the payload unit start indicator is set to 1 in the TS packet including the head of the access unit. In the MMTP packet that transmits the video, when the fragment_indicator is 00 or 01, the head of the NAL unit is identified, and when the header type of the NAL unit is the AU delimiter, the MMTP packet that includes the head of the access unit is identified. As a result, the payload unit start indicator is set to 1 in the first packet when the MMTP packet is converted into a TS packet.

In the MMTP packet for transmitting voice, the fragment_indicator is 00 or 01 to identify the head of the access unit, and the payload unit start indicator is set to 1 in the head packet when the MMTP packet is converted into a TS packet. .

When the MMTP packet is scrambled, it is descrambled and re-scrambled by the scramble method in the TS method.

When the MMTP packet is scrambled, a multi-type extension header is added to the extension header area in the MMTP packet, and if the multi-extension header type is an extension header for scrambling, whether the scramble key is even or not Information about whether it is a key or an odd key is stored.

When converting the MMTP packet header directly into the TS packet header, the scramble control bit in the extension header of the MMTP packet is converted into the transport scramble control bit in the TS packet. When the scramble extension header is not indicated in the MMTP packet, the transport scramble control bit in the TS packet is set to 0 without scrambling.

In the case of re-scramble, the first scramble method may be used instead of the second scramble method.

121 is a diagram showing a reception processing flow for reception using the reception device shown in FIG. 118F.

The receiving device 1860 receives signals transmitted by a plurality of different methods (a signal transmitted by the first method and a signal transmitted by the second method) (S3701).

Next, the receiving device 1860 determines whether the received signal is the first method or the second method (S3702).

When the received signal is the second method (the second method in S3702), the receiving device 1860 executes the process using the second method processing means A1864 (S3703).

On the other hand, if the received signal is the first method (the first method in S3702), the receiving device 1860 executes the process using the first method processing means A1861 (S3705).

Next, the receiving apparatus 1860 determines whether or not the output of the first method processing means A1861 is processed by the first method processing means B1862 or the second method processing means B1865 (S3706). .

When it is determined that the reception apparatus 1860 performs processing using the second system processing unit B 1865 (Yes in step S3706), the system conversion unit 1863 converts the first system into the second system (S3707).

The receiving apparatus 1860 converts the second system signal processed by using the second system processing means A 1864 and the system conversion means 1863 from the first system signal to the second system signal. All signals are processed using the second method processing means B1865 (S3704).

On the other hand, if it is determined that the receiving apparatus 1860 performs processing using the first method processing means B1862 (No in step S3706), the receiving apparatus 1860 executes processing using the first method processing means B1962 (S3708).

As described above, the receiving apparatus 1860 that receives signals of a plurality of different systems (the first system and the second system) includes the system conversion means 1863, so that signals of any system can be transmitted to the second system. Processing can be executed in processing means B1865.

Note that if there is no signal to execute processing using the first method processing means B, step S3706 and step S3708 may be omitted, and the output of step S3705 may be connected to step S3707.

FIG. 122 is a diagram showing a processing flow of the method conversion means 1920 in FIG.

The method conversion means 1920 acquires the signal output from the first method processing means A 1910 (S3801).

Next, it is determined whether or not the acquired signal is a signal for processing using the second system processing means B 1933 (S3802).

If the method conversion unit 1920 is a signal to be processed using the second method processing unit B 1933 (Yes in step S3802), the second method conversion unit 1922 converts the first method packet into the second method. The packet is converted (S3803). Then, the method conversion unit 1920 assigns an identifier for identifying a component, control information, and the like (S3804). In other words, the second system conversion unit 1922 gives the second identifier indicating the second packet (TS packet) obtained by the second conversion to the second packet of the second conversion data.

On the other hand, if the method conversion means 1920 is not a signal to be processed using the second method processing means B 1933 (No in step S3802), the first method conversion means 1921 sends the first method packet to the second method. Store in the packet (S3805). Further, the method conversion unit 1920 assigns an identifier indicating that the packet of the first method is stored in the packet of the second method (S3806). That is, the first scheme conversion means 1921 gives the first identifier indicating the second packet (TS packet) obtained by the first conversion to the second packet of the first conversion data.

The second scheme multiplexing means 1923 multiplexes the packets that have been packetized by the second scheme from step S3801 to step S3806 and given the identifier, and outputs the multiplexed packets to the scheme processing means B 1930 (S3807).

FIG. 123 is a diagram showing a processing flow of the system processing means B 1930 in FIG.

The system processing unit B 1930 acquires the signal multiplexed by the second system output from the system conversion unit 1920 (S3901).

Next, the scheme processing means B 1930 determines from the packet identifier whether or not the packet of the first scheme is a stored packet (S3902).

If the first method packet is a packet in which the first method packet is stored (Yes in step S3902), the method processing unit B 1930 extracts the first method packet and executes the process in the first method processing unit B 1932 ( S3903).

On the other hand, if the first method packet is not a stored packet (No in step S3902), the method processing unit B 1930 extracts the second method packet and executes the process in the second method processing unit B 1933. (S3904).

118C to 118F, the second method processing means

B

1833, 1844, 1854, and 1865 in the receiving devices 1830 to 1860 include second method transmission means as a retransmission unit (not shown), or A retransmitting apparatus may be configured by replacing the configuration. In this case, it is possible to retransmit the signal received by the first method using the second method.

That is, it is good also as a structure provided with the re-transmission part which retransmits the conversion data output by the system conversion means 1832, 1843, 1852, 1863 as a conversion part to another receiving apparatus.

It should be noted that the second method processing means

B

1833, 1844, 1854, and 1865 in the receiving apparatuses 1830 to 1860 in FIGS. 118C to 118F include or replace the second method storage means as a storage unit (not shown). The storage device may be configured with the above configuration. In this case, the signal received by the first method can be stored by the second method.

That is, it is good also as a structure provided with the accumulation | storage part which accumulate | stores the conversion data output by the system conversion means 1832, 1843, 1852, 1863 as a conversion part in a memory | storage device. The storage device is realized by, for example, an auxiliary storage device configured by a hard disk drive, a nonvolatile memory, or the like.

In addition, in the receiving apparatuses 1830 to 1860 in FIGS. 118C to 118F, the processing is performed by two processing means, that is, the processing on the front stage side and the processing on the rear stage side, but may be configured by three or more processing means. I explained. That is, as described above, in the case where there is a retransmitting unit, the second method processing means

B

1833, 1844, 1854, 1865 and the first method processing means 1842, 1862 on the rear stage side are not provided. Also good. That is, the second system processing means

B

1833, 1844, 1854, 1865 and the first system processing means 1842, 1862 on the subsequent stage side may be configured to be provided in another receiving apparatus.

Specifically, the signal transmitted from the second-type retransmission apparatus and the signal reproduced from the second-type storage apparatus are second type processing means B1833, 1844 provided in other receiving apparatuses, It can be played back at 1854, 1865. Further, the storage device may further include a retransmission unit, and the conversion data stored in the storage device may be retransmitted to another reception device. In addition to the method of accumulating signals after processing by the

method conversion units

1832, 1843, 1852, and 1863, when accumulating the signal of the first method before processing by the method conversion unit and reproducing the accumulated signal In addition, reproduction may be performed using the system conversion means and the second system processing means.

[Supplement: Receiver]
The receiving apparatus can be configured as shown in FIG. The receiving apparatus can be configured as shown in FIG. FIG. 124 is a diagram illustrating an example of a specific configuration of the reception apparatus. FIG. 125 is a diagram illustrating another example of the specific configuration of the reception device.

The receiving device 2100 includes a first processing unit 2101 and a conversion unit 2102. Each of the first processing unit 2101 and the conversion unit 2102 is realized by, for example, a microcomputer, a processor, or a dedicated circuit. That is, each processing unit constituting the receiving device 2100 may be realized by software or hardware.

The receiving apparatus 2200 includes a receiving unit 2201, a demultiplexing unit 2202, a first decoding unit 2203, a second decoding unit 2204, and an output unit 2205. Each of the receiving unit 2201, the demultiplexing unit 2202, the first decoding unit 2203, the second decoding unit 2204, and the output unit 2205 is realized by, for example, a microcomputer, a processor, or a dedicated circuit. That is, each processing unit constituting receiving device 2200 may be realized by software or hardware.

Detailed description of each component of the receiving

devices

2100 and 2200 will be given in the description of the corresponding receiving method.

First, the reception method of the reception device 2100 will be described with reference to FIG. FIG. 126 is an operation flow (reception method) by the reception apparatus.

First, the first processing unit 2101 of the reception apparatus 2100 includes first multiplexed data of the first multiplexing scheme (MMT / TLV scheme) and a second multiplexing scheme (TS scheme) different from the first multiplexing scheme. 2 receives a broadcast signal in which multiplexed data composed of at least first multiplexed data of two multiplexed data is modulated, demodulates the received broadcast signal, and outputs multiplexed data obtained by demodulating ( S4001).

Next, the conversion unit 2102 of the reception apparatus 2100 converts the multiplexed method of the first multiplexed data from the output multiplexed data to the second multiplexed method, and outputs the converted data obtained by the conversion. (S4002).

Thus, if the data converted into the second multiplexing method by the receiving device 2100 is used, the conventional TS processing unit can be used as it is. For this reason, in subsequent processing, sharing with the existing mounting can be easily realized, and can be realized at low cost.

Next, a reception method of the reception device 2200 will be described with reference to FIG. FIG. 127 is an operation flow (reception method) by the reception apparatus.

First, the receiving unit 2201 of the receiving device 2200 is a second packet (TS packet) in which a first packet (MMTP packet) constituting first multiplexed data of the first multiplexing method (MMT / TLV method) is stored. Thus, the first conversion data composed of the second packet used in the second multiplexing method (TS method) different from the first multiplexing method, and the first packet being converted into the second multiplexing method. The conversion data multiplexed with the second conversion data composed of the second packet obtained in step S4101 is received (S4101).

Next, the demultiplexing unit 2202 of the receiving device 2200 performs a demultiplexing process for demultiplexing the converted data received by the receiving unit 2201 into the first converted data and the second converted data (S4102).

Next, first decoding section 2203 of receiving apparatus 2200 extracts the first packet from the second packet that constitutes the first converted data obtained by the demultiplexing process, and the extracted first packet constitutes the first packet. A first decoding process using the first multiplexing method is performed on the first data (S4103).

In addition, the second decoding unit 2204 of the receiving device 2200 performs the second multiplexing method on the second data included in the second packet constituting the second converted data obtained by the demultiplexing process. 2 Decode processing is performed (S4104).

Finally, the output unit 2205 of the receiving device 2200 outputs the first decoded data obtained by the first decoding process and the second decoded data obtained by the second decoding process (S4105).

Thus, since the receiving device 2200 can use the data converted into the second multiplexing method, for example, the conventional TS processing unit can be used as it is. For this reason, the receiving apparatus 2200 can be easily shared with the existing mounting, and can be realized at low cost.

(Other embodiments)
Although the transmission apparatus, the reception apparatus, the transmission method, and the reception method according to the embodiment have been described above, the present disclosure is not limited to this embodiment.

Further, each processing unit included in the transmission device and the reception device according to the above embodiment is typically realized as an LSI that is an integrated circuit. These may be individually made into one chip, or may be made into one chip so as to include a part or all of them.

Further, the integration of circuits is not limited to LSI, and may be realized by a dedicated circuit or a general-purpose processor. An FPGA (Field Programmable Gate Array) that can be programmed after manufacturing the LSI or a reconfigurable processor that can reconfigure the connection and setting of circuit cells inside the LSI may be used.

In each of the above embodiments, each component may be configured by dedicated hardware or may be realized by executing a software program suitable for each component. Each component may be realized by a program execution unit such as a CPU or a processor reading and executing a software program recorded on a recording medium such as a hard disk or a semiconductor memory.

In other words, the transmission device and the reception device include a processing circuit and a storage device (storage) that is electrically connected to the processing circuit (accessible from the control circuit). The processing circuit includes at least one of dedicated hardware and a program execution unit. Further, when the processing circuit includes a program execution unit, the storage device stores a software program executed by the program execution unit. The processing circuit executes the transmission method or the reception method according to the above embodiment using the storage device.

Further, the present disclosure may be the above software program or a non-transitory computer readable recording medium on which the above program is recorded. Needless to say, the program can be distributed via a transmission medium such as the Internet.

Further, all the numbers used above are examples for specifically explaining the present disclosure, and the present disclosure is not limited to the illustrated numbers.

In addition, division of functional blocks in the block diagram is an example, and a plurality of functional blocks can be realized as one functional block, a single functional block can be divided into a plurality of functions, or some functions can be transferred to other functional blocks. May be. In addition, functions of a plurality of functional blocks having similar functions may be processed in parallel or time-division by a single hardware or software.

In addition, the order in which the steps included in the transmission method or the reception method are executed is for illustration in order to specifically describe the present disclosure, and may be in an order other than the above. Also, some of the above steps may be executed simultaneously (in parallel) with other steps.

Although the transmission device, the reception device, the transmission method, and the reception method according to one or more aspects of the present disclosure have been described based on the embodiments, the present disclosure is not limited to the embodiments. Absent. Unless it deviates from the gist of the present disclosure, one or more of the present disclosure may be applied to various modifications conceived by those skilled in the art in this embodiment, or a combination of components in different embodiments. It may be included within the scope of the embodiments.

This disclosure can be applied to an apparatus or device that performs media transport such as video data and audio data.

15, 100, 300, 500, 700, 900, 1200, 1400

Transmitter

16, 101, 301

Encoder

17, 102

Multiplexer

18, 104

Transmitter

20, 200, 400, 600, 800, 1000, 1300, 1500, 1810, 1820, 1830, 1840, 1850, 1860, 2100, 2200 Receiver 21 Packet filtering unit 22 Transmission order type discriminating unit 23

Random access unit

24, 212 Control information acquisition unit 25 Data acquisition unit 26 Calculation unit 27 Initialization

Information acquisition unit

28, 206

Decoding instruction unit

29, 204A, 204B, 204C, 204D, 402 Decoding unit 30 Presentation unit 201 Tuner 202 Demodulation unit 203 Demultiplexing unit 205 Display unit 211 Type discrimination unit 213 Slice information acquisition unit 214 Decoding data Data generating unit 302 adding unit 303,503,702,902,1202,1402 transmitting unit 401,601,801,1001,1301,1501,201 receiving unit 501 dividing unit 502,603 component unit 602 determining unit 701,901 1201, 1401 generation unit 802 first buffer 803 second buffer 804 third buffer 805 fourth buffer 806 decoding unit 1002 reproduction unit 1302 calculation unit 1303 correction unit 1502

determination units

1811, 1821, 1831, 1841, 1851, 1861 First method processor A
1812, 1822, 1842, 1862 First method processing means B
1823, 1853, 1864 Second method processing means A
1824, 1833, 1844, 1854, 1865 Second method processing means B
1832, 1843, 1852, 1863 System conversion means 2101 First processing unit 2102 Conversion unit 2202 Demultiplexing unit 2203 First decoding unit 2204 Second decoding unit 2205 Output unit

Claims

Multiplexed data composed of at least the first multiplexed data among the first multiplexed data of the first multiplexing scheme and the second multiplexed data of the second multiplexing scheme different from the first multiplexing scheme is A first processor that receives the modulated broadcast signal, demodulates the received broadcast signal, and outputs the multiplexed data obtained by demodulating;
A conversion unit that converts the multiplexing method of the first multiplexed data out of the output multiplexed data into the second multiplexing method and outputs the converted data obtained by the conversion;
A receiving device.
The converter is
A first conversion for extracting a part of the first data of the first multiplexed data and storing a first packet constituting the first data in a second packet used in the second multiplexing method. Performing a first conversion unit configured to output first conversion data composed of the second packet obtained by the first conversion;
A second packet which extracts the first packet constituting the second data of the remaining part of the first multiplexed data and converts the extracted first packet into a second packet of the second multiplexing method; A second conversion unit that performs conversion and outputs second conversion data composed of the second packet obtained by the second conversion;
A multiplexing unit that performs a multiplexing process for multiplexing the output first converted data and the second converted data;
The receiving apparatus according to claim 1, wherein data obtained by the multiplexing process is output as the converted data.
The receiving device according to claim 2, wherein the first data and the second data are data of different media.
The first conversion unit assigns a first identifier indicating the second packet obtained by the first conversion to the second packet of the first conversion data;
The receiving apparatus according to claim 2, wherein the second conversion unit assigns a second identifier indicating that the second packet is obtained by the second conversion to the second packet of the second conversion data.
The receiving device according to claim 1, further comprising a retransmitting unit that retransmits the converted data output by the converting unit to another receiving device.
The receiving device according to claim 1, further comprising: an accumulation unit that accumulates the converted data output by the conversion unit in a storage device.
The receiving device according to claim 6, further comprising a retransmission unit that retransmits the converted data stored in the storage device to another receiving device.
The receiving apparatus according to claim 1, further comprising: a second processing unit that performs a decoding process for decoding the converted data output by the conversion unit and outputs the decoded data obtained by the decoding process.
Furthermore, a decoding process for decoding the conversion data output by the conversion unit is provided, and a second processing unit for outputting the decoded data obtained by the decoding process is provided.
The second processing unit includes:
A demultiplexing unit that performs a demultiplexing process that demultiplexes the converted data output by the converting unit into the first converted data and the second converted data;
In the first multiplexing method, the first data constituting the first packet extracted from the second packet constituting the first converted data obtained by the demultiplexing process is used. A first decoding unit for performing one decoding process;
A second decoding for performing a second decoding process in the second multiplexing method on the second data constituting the second packet constituting the second converted data obtained by the demultiplexing process; And
The receiving apparatus according to claim 2, wherein the first decoding data obtained by the first decoding process and the second decoding data obtained by the second decoding process are output as the decoding data.
The receiving device further includes:
An adjustment unit configured to adjust one of the first control information and the second control information to the other by using the first control information of the first decode data and the second control information of the second decode data; The receiving device according to claim 9.
The first control information is first reference clock information,
The second control information is second reference clock information,
The adjustment unit synchronizes the first decode data and the second decode data by performing adjustment to adjust one of the first reference clock information and the second reference clock information to the other. Receiver device.
The first multiplexing scheme is an MMT / TLV scheme,
The receiving apparatus according to claim 1, wherein the second multiplexing scheme is a TS scheme.
A second packet in which a first packet constituting the first multiplexed data of the first multiplexing method is stored, and is composed of a second packet used in a second multiplexing method different from the first multiplexing method. Receiving the converted data in which the first converted data and the second converted data composed of the second packet obtained by converting the first packet into the second multiplexing method are multiplexed. A receiver,
A demultiplexing unit that performs a demultiplexing process that demultiplexes the converted data received by the receiving unit into the first converted data and the second converted data;
The first packet is extracted from the second packet that constitutes the first converted data obtained by the demultiplexing process, and the first data with respect to the first data that the extracted first packet comprises A first decoding unit for performing a first decoding process in a multiplexing scheme;
A second decoding unit that performs a second decoding process in the second multiplexing method on the second data that is constituted by the second packet that constitutes the second converted data obtained by the demultiplexing process When,
An output unit for outputting the first decoded data obtained by the first decoding process and the second decoded data obtained by the second decoding process;
A receiving device.
Multiplexed data composed of at least the first multiplexed data of the first multiplexed data of the first multiplexing scheme and the second multiplexed data of the second multiplexing scheme different from the first multiplexing scheme is modulated. Receiving the broadcast signal, demodulating the received broadcast signal, outputting the multiplexed data obtained by demodulating,
A receiving method for converting the multiplexed system of the first multiplexed data from the output multiplexed data to the second multiplexed system and outputting the converted data obtained by the conversion.
A second packet in which a first packet constituting the first multiplexed data of the first multiplexing method is stored, and is composed of a second packet used in a second multiplexing method different from the first multiplexing method. Receiving the converted data in which the first converted data and the second converted data composed of the second packet obtained by converting the first packet into the second multiplexing method are multiplexed. ,
Performing a demultiplexing process for demultiplexing the received converted data into the first converted data and the second converted data;
The first packet is extracted from the second packet that constitutes the first converted data obtained by the demultiplexing process, and the first data with respect to the first data that the extracted first packet comprises Perform the first decoding process in the multiplexing system,
Performing a second decoding process in the second multiplexing method on the second data constituting the second packet constituting the second converted data obtained by the demultiplexing process;
A receiving method for outputting first decoded data obtained by the first decoding process and second decoded data obtained by the second decoding process.